# Gaspatchio

Gaspatchio is an actuarial modelling framework that allows you to build and run actuarial models in pure Python but built for use with LLMs from the ground up.

# Get started

# Gaspatchio

Gaspatchio is a Python actuarial modelling framework. You build your projection vectors — survival, account values, cashflows, reserves — as columns on an `ActuarialFrame`. The whole projection runs across every policy together, and every output traces back to the inputs and steps that produced it.

## What makes it different

**You write the projection, not the loop.** Assignments like `af.qx = mortality.lookup(...)` or `af.survival = af.qx.projection.cumulative_survival()` return a vector across the projection — automatically, per policy. The same line of model code works for one policy or a million.

**Closed-form cumulative for most things, rollforward for the rest.** Survival probabilities, discount factors, account values with pre-computable charges — closed-form cumulative operations on list columns. COI on net amount at risk, IUL floor/cap, GMDB ratchet, multi-state products — a `rollforward()` state machine that reads like the within-period product spec.

**Fast on the laptop you've already got.** The model code is Python; the execution is Rust. A 100,000-policy run with monthly projections over 30 years — mortality lookups, survival, cashflows, account values — finishes in seconds on a workstation, not overnight on a cluster. A million-policy run is minutes, not hours. Edit the model, run it, see the numbers — no batch queue, no IT ticket, no separate environment for "the real run".

**Auditable end-to-end.** Every output column traces back to inputs and intermediate steps. Schedules, assumption tables, curves, mortality tables, and compiled rollforwards each carry a `source_sha()`, and that identity rolls up into the rollforward's overall version stamp — so quarter-over-quarter drift is visible the moment it appears, not on page seventeen of an experience study.

## A short example

Three policies, five-year projection. Look up monthly mortality rates against per-policy attained-age vectors, convert to cumulative survival, and produce an expected-claim vector per policy — all in one composed plan:

```python
import polars as pl
from gaspatchio import ActuarialFrame
from gaspatchio.assumptions import Table

af = ActuarialFrame(
    pl.DataFrame(
        {
            "policy_id": ["P001", "P002", "P003"],
            "sum_assured": [100_000.0, 250_000.0, 500_000.0],
            "attained_age": [
                [35, 36, 37, 38, 39],
                [42, 43, 44, 45, 46],
                [55, 56, 57, 58, 59],
            ],
        }
    )
)

mortality = Table(
    name="qx_demo",
    source=pl.DataFrame(
        {
            "age": list(range(30, 76)),
            "qx": [0.001 + (a - 30) * 0.0008 for a in range(30, 76)],
        }
    ),
    dimensions={"age": "age"},
    value="qx",
)

# Vector lookup against the attained-age vector — one rate vector per policy
af.qx = mortality.lookup(age=af.attained_age)

# Closed-form cumulative survival — replaces the recursive ₜpₓ loop
af.survival = af.qx.projection.cumulative_survival()

# Expected claims per policy per year, as a vector
af.expected_claims = af.survival * af.qx * af.sum_assured

print(af.select("policy_id", "survival", "expected_claims").collect())
```

```text
shape: (3, 3)
┌───────────┬───────────────────────────┬─────────────────────────────────┐
│ policy_id ┆ survival                  ┆ expected_claims                 │
│ ---       ┆ ---                       ┆ ---                             │
│ str       ┆ list[f64]                 ┆ list[f64]                       │
╞═══════════╪═══════════════════════════╪═════════════════════════════════╡
│ P001      ┆ [1.0, 0.995, … 0.975428]  ┆ [500.0, 577.1, … 799.851049]    │
│ P002      ┆ [1.0, 0.9894, … 0.953627] ┆ [2650.0, 2819.79, … 3290.01427… │
│ P003      ┆ [1.0, 0.979, … 0.914112]  ┆ [10500.0, 10671.1, … 11060.755… │
└───────────┴───────────────────────────┴─────────────────────────────────┘
```

`mortality.lookup` returns a five-element vector per policy, aligned to that policy's attained-age vector. `projection.cumulative_survival()` walks each vector once. `expected_claims` broadcasts the scalar `sum_assured` against the survival and qx vectors. Nothing in the model code mentions "time step" or "iteration" — and the plan runs across every policy together when `.collect()` is called.

## Where to go next

- **[Tutorials](https://gaspatchio.dev/0.9.0/tutorials/index.md)** — build a model on the four-level ladder, from a first projection through a reconciled production model.
- **[Concepts](https://gaspatchio.dev/0.9.0/concepts/intro/index.md)** — understand how the framework thinks. Start with the Intro, then Schedules, Projections, Assumptions, and Rollforward.
- **[API Reference](https://gaspatchio.dev/0.9.0/api/intro/index.md)** — full surface: `ActuarialFrame`, `Schedule`, `Table`, `MortalityTable`, `Curve`, `rollforward`, `accumulate`, scenarios, and the Excel/finance/date function namespaces.

Already working with an AI coding assistant?

Paste this into its chat — it can set up Gaspatchio for you:

```text
Run `uv add gaspatchio` (or `pip install gaspatchio`) to install Gaspatchio,
then set up its plugin for this editor by following
https://gaspatchio.dev/ai/setup/, then verify with `gspio tutorial list`.
```

Installation is as simple as:

```bash
uv add gaspatchio
```

New to uv?

[uv](https://docs.astral.sh/uv/) is a fast Python package and project manager. Install it once and the command above will work:

```bash
# macOS / Linux
curl -LsSf https://astral.sh/uv/install.sh | sh
```

```powershell
# Windows (PowerShell)
powershell -c "irm https://astral.sh/uv/install.ps1 | iex"
```

Prefer to use a Python you already have? `pipx install uv` (or `pip install uv`) works too. Full options are in the [uv installation guide](https://docs.astral.sh/uv/getting-started/installation/).

```bash
pip install gaspatchio
```

## Check it worked

Gaspatchio installs as the **`gaspatchio`** package but imports as **`gaspatchio`**:

```bash
python -c "from gaspatchio import ActuarialFrame; print('gaspatchio ready')"
```

With uv, prefix the command with `uv run`:

```bash
uv run python -c "from gaspatchio import ActuarialFrame; print('gaspatchio ready')"
```

## Set up the plugin (recommended)

For best results, model with an AI coding tool

Gaspatchio is built for AI-powered modelling. Because a model is plain, explicit Python — no proprietary language, no hidden calculations — you can use an AI coding tool to generate, debug, and reconcile gaspatchio models from a plain-language description, then verify every calculation against your existing models. The plugin equips your editor with the framework's patterns, CLI commands, and performance guardrails, so generated code follows gaspatchio's conventions instead of guessing at the API.

In Claude Code, register the marketplace and install the plugin:

```text
/plugin marketplace add gaspatchio/gaspatchio
/plugin install gaspatchio@gaspatchio
```

Using VS Code, Cursor, or another agent? The two-minute setup for each is on the [Plugins](https://gaspatchio.dev/0.9.0/ai/setup/index.md) page.

## Install from source

To build the latest directly from the public repository (a Rust toolchain is required to compile the core):

```bash
pip install "git+https://github.com/gaspatchio/gaspatchio.git#subdirectory=bindings/python"
```

## Install from a release wheel

Download the wheel for your platform from the [GitHub releases page](https://github.com/gaspatchio/gaspatchio/releases), then install the file you downloaded (the version and platform tag will match your download — the line below is only an example):

```bash
uv pip install gaspatchio-0.5.1-cp313-cp313-macosx_11_0_arm64.whl
```

# Tutorials

A progressive tutorial for building actuarial models with Gaspatchio. Each level introduces new concepts while producing a working model you can run and inspect.

The tutorial is designed for actuaries who know Excel and are learning Python-based modelling. The models themselves teach the Python — you don't need prior Python experience to start.

## Levels

| Level                                                       | Name                                                                                                                           | What You'll Learn                             | Start Here If... |
| ----------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------- | ---------------- |
| [1 — Hello World](#level-1-hello-world)                     | `ActuarialFrame`, column arithmetic, `when/then/otherwise`, `.collect()`                                                       | You're new to Gaspatchio                      |                  |
| [2 — Assumptions](#level-2-assumptions)                     | `Table.lookup()`, multi-dimension tables, loading from files                                                                   | You know DataFrames but not Gaspatchio Tables |                  |
| [3 — Mini Variable Annuity](#level-3-mini-va)               | Full VA projection: mortality, lapse, AV, claims, discounting. 6 incremental steps.                                            | You want the full picture first               |                  |
| [4 — Reconciled Lifelib Model](#level-4-reconciled-lifelib) | Production model reconciled against lifelib IntegratedLife at 0.0000% across 1,016 model points, 35 variables, ~9M data points | You're ready to build a production model      |                  |
| [5 — Scenarios](#level-5-scenarios)                         | Deterministic scenarios, parameter shocks, sensitivity sweeps, regulatory-style reports with Altair charts                     | You need scenario analysis                    |                  |

## Using the Tutorials with AI

If you have the [gaspatchio plugin installed](https://gaspatchio.dev/0.9.0/ai/setup/index.md), you can work through these tutorials using natural language. Open your editor and ask:

- *"Start me on the Level 1 Hello World tutorial."*
- *"I've done Level 2 — take me to Level 3."*
- *"I want to reconcile a model against lifelib. Which level do I need?"*
- *"Guide me through Level 3 step by step."*

The tooling loads the tutorial content, presents each step in actuarial terms, and supports you in running and modifying the models at your own pace.

## Quick Start

Tutorials ship with the gaspatchio package. Use the CLI to get started:

```bash
# See what's available
gspio tutorial list

# Initialize Level 1 into your working directory
gspio tutorial init level-1

# Run it
cd my-first-model
uv run python model.py

# Verify the output matches expected
gspio tutorial verify level-1
```

**New to Gaspatchio?** Start with Level 1 — it's 60 lines and covers the four core concepts.

**Want a complete VA model to study?** Initialize Level 3 — a full variable annuity projection in a single file with inline data:

```bash
gspio tutorial init level-3 --dest ./va-model
```

Read the docstring, read the code section by section, then run it.

______________________________________________________________________

## Level 1: Hello World

Three term life insurance policies, no prior Python experience required. The base model is about 60 lines and introduces:

- **ActuarialFrame** — the central data structure
- **Column arithmetic** — operators work on entire projection vectors
- **`when/then/otherwise`** — conditional logic for actuarial rules
- **`.collect()`** — run the full calculation when you are ready

| Step               | What It Adds                                  |
| ------------------ | --------------------------------------------- |
| Base               | 3 policies, scalar arithmetic, no projections |
| 01 — Projections   | Add time dimension (list columns)             |
| 02 — Survival      | Cumulative survival probability               |
| 03 — Time Shifting | `previous_period()`, per-period deaths        |

```bash
# Initialize and run the base model
gspio tutorial init level-1
cd my-first-model && uv run python model.py
```

## Level 2: Assumptions

Teaches assumption table lookups — the core mechanism for separating business assumptions from model logic.

| Step                 | What It Adds                          |
| -------------------- | ------------------------------------- |
| Base                 | Mortality from Table (age lookup)     |
| 01 — Multi-Dimension | Age x sex mortality Table             |
| 02 — From Files      | Load data from parquet files          |
| 03 — Lapse           | Lapse rate Table, combined decrements |
| 04 — Conditionals    | `when/then/otherwise` on list columns |

```bash
gspio tutorial init level-2
cd my-first-model && uv run python model.py
```

## Level 3: Mini VA

A complete variable annuity projection built incrementally. The base model uses inline data — no external files needed. Each step adds one feature so you can see exactly what changed and why.

| Step                  | What It Adds                                                                                                 |
| --------------------- | ------------------------------------------------------------------------------------------------------------ |
| Base                  | Inline data, simplest VA model                                                                               |
| 01 — From Files       | Data from parquet files                                                                                      |
| 02 — Select Mortality | Select/ultimate mortality tables, multi-key lookup, mortality scalars                                        |
| 03 — Guarantees       | GMDB, GMAB guarantees, surrender charges, nested conditionals                                                |
| 04 — Dynamic Lapse    | ITM ratio, dynamic lapse formula, duration-based lapse tables                                                |
| 05 — Rate Curves      | Risk-free rate curve discounting, term-structure lookup                                                      |
| 06 — Reconcile        | Bridge to Level 4: 4 actuarial gaps discovered and fixed via reconciliation against lifelib                  |
| 07 — Rollforward      | Replace `cum_prod()` AV growth with the rollforward state-machine kernel — same numbers, explicit recurrence |

```bash
# Initialize and run the base model (inline data, self-contained)
gspio tutorial init level-3 --dest ./va-model
cd va-model && uv run python model.py
```

## Level 4: Reconciled Lifelib

The full appliedlife model, reconciled against lifelib's IntegratedLife implementation. Projects monthly cashflows and present values for GMDB/GMAB variable annuity products.

**Key achievement:** 0.0000% difference across 1,016 model points (IF + NB), 35 variables, ~9M data points.

Uses `accumulate()` for the AV accumulation. The model is ~860 lines with 14 assumption table files.

```bash
# Initialize and run the model
gspio tutorial init level-4 --dest ./lifelib-model
cd lifelib-model && uv run gspio run-model model.py model_points.parquet
```

## Level 5: Scenarios

Scenario analysis for the reconciled VA model — every step uses the typed `ScenarioRun` plan and the bounded-memory `for_each_scenario` loop. Each plan has a stable SHA, an audit sidecar, and produces per-scenario aggregates ready for tornado / sensitivity / heatmap visualisation.

| Step                        | What It Adds                                                                                                                                                                                        |
| --------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Base                        | Interest-rate scenarios (BASE/UP/DOWN) via `with_scenarios`, scenario-aware `Curve` discounting                                                                                                     |
| 01 — Parameter Shocks       | `ScenarioRun` plan, `MultiplicativeShock` against assumption tables, mergeable aggregators with `.over("scenario_id")`, audit sidecar, tornado data                                                 |
| 02 — Conditional Shocks     | `FilteredShock` (e.g. age ≥ 65), `TimeConditionalShock` (e.g. duration ≤ 5), `PipelineShock` (e.g. Solvency II ×1.5 then clip 1.0)                                                                  |
| 03 — Sensitivity            | 1D sweep (mortality multiplier across a range) and 2D mortality × lapse interaction grid for heatmap visualisation — built as plain `ScenarioRun` plans, no special helper                          |
| 04 — Scenario Comparison    | Named regulatory stresses (`CENTRAL_ESTIMATE`, `ADVERSE_MORTALITY`, `ECONOMIC_DOWNTURN`, `OPERATIONAL_SHOCK`), full audit chain (`source_sha`, `to_yaml`, JSON audit sidecar), reproducibility memo |
| 05 — Stochastic Monte Carlo | 200 paths via `master_seed`; `drivers["rng_seed"]` flows to the model; `CTE` + `Quantile` aggregators give SCR-shape capital figures; same seed → same SHA → byte-identical aggregates              |

```bash
# Initialize and run base scenarios
gspio tutorial init level-5 --dest ./scenarios-model
cd scenarios-model && uv run python run_scenarios.py
```

______________________________________________________________________

## Running Models

Initialize any tutorial with `gspio tutorial init`, then run it:

**Standalone Python** — run the model directly:

```bash
uv run python model.py
```

**Single policy** — debug one policy, see every variable:

```bash
uv run gspio run-single-policy model.py data.parquet 1
```

**All policies** — run the full model point set:

```bash
uv run gspio run-model model.py data.parquet
```

**Verify output** — confirm the model produces expected results:

```bash
gspio tutorial verify level-1
```

# Principles

These are the positions the framework takes. They are not features and not selling points — they are the choices we made and the alternatives we rejected. If you disagree with one of these, the framework will feel wrong in ways no tutorial will fix. If you agree, you'll find the framework gets out of your way.

## Meet you where you are

You arrive at any new modelling tool with a working setup: tables in the shape someone published them in, formulas you know from spreadsheets, a Python skill you didn't pick up to use this framework specifically. The framework that demands you reshape your inputs before it'll talk to you, learn its proprietary DSL, or build an ETL pipeline before you write a single line of model code is the framework that gets shelved. We accept wide mortality tables, select-ultimate tables with "Ult." columns, registry conventions you didn't invent. Formulas read like spreadsheets or pure mathematical expressions, not like programs. The framework reshapes itself to you, not the other way round.

## Closed-form by default, escalation when state demands it

Recursive cell-graphs — function calling function with time offsets, evaluation order resolved at run time — were the answer when actuarial modelling first landed in Python. They are not the answer now. Most accumulations you care about have a closed form: survival probabilities are a cumulative product, discount factors are a power, account value with pre-computable charges is a linear recurrence. These run faster, read clearer, and don't require you to mentally unwind a recursion to verify a single line. We reach for the state-machine rollforward only when within-period charges actually depend on the running balance — COI on net amount at risk, IUL floor/cap, GMDB ratchet. The escalation is deliberate and explicit, not the default.

## Optimize for the laptop, not the cluster

Most actuarial modelling stacks scale by running somewhere else — overnight on a vendor batch, on a cluster IT provisioned six months ago, in a "production environment" you don't develop in. Gaspatchio targets the machine on your desk. A 100,000-policy run with monthly projections over 30 years finishes in seconds on a laptop; a million-policy run is minutes. That isn't a marketing claim about scaling up — it's a constraint we work to. The edit-run-refine loop slipping past a few seconds is the failure mode we treat as broken, and we would rather drop a capability than loosen the loop to add it.

## Audit by default, not by request

Governance is what someone bolts on after the framework is chosen, the model is built, the regulator has asked a question. We invert that order. Every output column carries expression lineage that traces back through every intermediate step. Every Schedule, Table, Curve, MortalityTable, and compiled rollforward carries an identity stamp — `source_sha()` or `fingerprint()` — that changes the moment the underlying source does. The model is auditable because we built it that way, not because you ran a separate workflow to make it so. Quarter-over-quarter drift becomes visible the moment it appears, not on page seventeen of an experience study, not in the meeting where everyone has already left.

## LLM-shaped from the inside out

You and an LLM working together is the design target, not a retrofit. The shape of the documentation — structured, executable, with a "When to use" section in actuarial language tied to runnable scalar and vector examples — exists because that's what makes retrieval hit the right answer. The shape of the error messages — actuarial concept, next move, in plain English — exists because that's what lets the next attempt converge. The shape of the retrieval surface — `gspio docs` and `gspio knowledge` as CLI commands, not an MCP server — was chosen for token efficiency, error handling, and the LLM's training data on standard CLI conventions. The agentic plugins for Claude Code, Cursor, and VS Code are the surface that uses this stack from inside the editor. Every choice points at the same target.

## Sharp knives, no magic

Frameworks that wrap calculations in decorators, DSL macros, or magic class-attribute resolution save typing but cost transparency. The reviewer or the regulator who asks "where exactly is this calculation defined?" needs an answer that isn't "it's derived from the framework's behaviour around this field name." We expose primitives; you compose them. The composition is in the code, in plain Python, with the columns visible. When the composition is wrong, the framework refuses to run rather than silently filling in a fallback that looks right until the regulator asks. Sharp knives in a competent hand are safer than blunt knives that pretend not to cut.

## No vendor lock-in, by construction

The model you build in this framework is not a hostage. Open source isn't a tagline we use because it polls well — it's a structural commitment to your professional independence. The calculation is in the code. The data is in Polars and Parquet. The model file is Python. There is no proprietary file format that requires a licensed reader. There is no run-time on a vendor's server that disappears when the contract lapses. There is no commercial decision in someone else's boardroom that ends the ability to run last year's valuation.

# Why Gaspatchio?

Actuarial models live in three places today: spreadsheets that don't scale or version-control, vendor systems that lock the calculation behind a UI you don't own, and bespoke Python that takes an engineering team to keep performant. Gaspatchio is the Python option without the engineering-team requirement.

## What you get

**The model is Python; the execution is Rust.** Write models in standard Python syntax, with standard libraries and a standard debugger. The execution runs on a Rust engine — portfolio-scale runs (100,000 policies × 30 years monthly) finish in seconds on the laptop you have, not overnight on a cluster.

**Closed-form cumulative ops, plus a rollforward when you need state.** For the calculations that fit the shape — survival probabilities, discount factors, AV with pre-computable charges — closed-form cumulative operations on list columns replace the recursive cell-graph. For the ones that don't — COI on net amount at risk, IUL floor/cap, GMDB ratchet, multi-state products — a `rollforward()` chain that reads like the within-period spec.

**Auditable by default.** Every output column traces back to inputs and intermediate steps. Schedules, assumption tables, curves, mortality tables, and compiled rollforwards each have an identity stamp — quarter-over-quarter drift is visible the moment it appears, not on page seventeen of an experience study.

**Actuary-plus-LLM workflow that actually works.** Python-native code is easy for AI coding tools to generate. Explicit calculations leave nothing for the LLM to hallucinate around. Error messages come back in actuarial language, and from your editor, `gspio docs` and `gspio knowledge` look up API methods and actuarial concepts directly — so the LLM working with you produces code you can review and sign off on, not a confidently-wrong intern.

**No vendor lock-in.** Open-source. The calculation is visible in the code — no proprietary file format, no licensing renewal, no separate run-time you don't control.

## When not to use it

- You need a GUI-driven modelling environment for non-technical users.
- Your team is committed to R as its stack.
- You need a vendor with a ready library of jurisdiction-specific products today — Gaspatchio gives you the framework; you build the product layer on top.

## Adjacent open-source projects

Other open-source actuarial work worth knowing about:

- **[lifelib](https://lifelib.io/)** + **[modelx](https://docs.modelx.io/)** — life-insurance modelling in Python via a recursive cell-graph approach.
- **[Heavylight](https://lewisfogden.github.io/heavylight/)** — lightweight Python projections using recursive syntax with caching.
- **[chainladder-python](https://chainladder-python.readthedocs.io/)** — P&C reserving in Python.
- **[OasisLMF](https://oasislmf.org/)** — open framework for catastrophe modelling.
- **[JuliaActuary](https://github.com/JuliaActuary)** — actuarial-science packages for Julia.
- **[Actuarial R packages](https://github.com/topics/actuarial)** — R packages across actuarial tasks.

## Where to go next

- **[Tutorials](https://gaspatchio.dev/0.9.0/tutorials/index.md)** — build a real model on the four-level ladder.
- **[Concepts](https://gaspatchio.dev/0.9.0/concepts/intro/index.md)** — understand how the framework thinks.
- **[Modelling with AI](https://gaspatchio.dev/0.9.0/ai/intro/index.md)** — the actuary-plus-LLM workflow in detail.

# No Soup For You

Actually — it's not *just* for you. Gaspatchio is built for actuaries working alongside LLMs, not actuaries replaced by them. The framework is structured so an LLM picking up the codebase produces actuarial code you can read, verify, and sign off on — which is what makes the actuary-plus-LLM workflow tractable instead of a confidently-wrong intern.

## Why it works

- **Python-native, no proprietary language.** AI coding tools are trained on Python; generated code is idiomatic, not a best-guess translation from a vendor DSL.
- **Explicit, auditable calculations.** Every formula is visible in the code — no hidden framework magic for an LLM to hallucinate around.
- **Errors come back in actuarial terms.** When code doesn't work, the runtime feedback is in actuarial language — so the next attempt converges instead of thrashing.
- **Designed for retrieval.** From your editor, `gspio docs` and `gspio knowledge` look up API methods and actuarial concepts directly, so the LLM working with you doesn't have to guess.

[**Modelling with AI →**](https://gaspatchio.dev/0.9.0/ai/intro/index.md) — six areas of modelling expertise, framework knowledge always loaded, searchable knowledge store, ready-to-invoke workflows.
# Concepts documentation

# Assumptions

## Tables Carry the Conventions They Were Built Under

An assumption table is more than rows of numbers. It carries a shape (wide or tidy), the dimensions you'll look it up by, an overflow convention for "Ultimate" or "Term" columns, and an identity that pins down which version of the table fed your valuation. The `Table` API records those conventions at registration so a lookup at projection time can't drift from the table's intended use.

For domain-shaped assumptions — mortality with select/ultimate and age-basis conventions, economic curves with term structure and key-rate stresses — Gaspatchio provides structure-aware objects (`MortalityTable`, `Curve`) that encode the actuarial conventions on top of (or alongside) the basic `Table`. Those are covered in [Mortality Tables](https://gaspatchio.dev/0.9.0/concepts/mortality/index.md) and [Curves](https://gaspatchio.dev/0.9.0/concepts/curves/index.md). This page is the underlying `Table` machinery they sit on, plus the assumption types that don't have a dedicated wrapper (lapse, expense, premium rates, sensitivity multipliers).

## What a Table Is

A `Table` is a named, dimensioned object the actuary registers once and looks up against at any point during the projection.

```python
import polars as pl
from gaspatchio import ActuarialFrame
from gaspatchio.assumptions import Table

lapse_df = pl.DataFrame(
    {
        "duration": [1, 2, 3, 4, 5],
        "lapse_rate": [0.08, 0.06, 0.04, 0.03, 0.02],
    }
)

lapse = Table(
    name="lapse_rates",
    source=lapse_df,
    dimensions={"duration": "duration"},
    value="lapse_rate",
)

af = ActuarialFrame(pl.DataFrame({"policy_id": ["P001", "P002"], "duration": [2, 4]}))
af.lapse_rate = lapse.lookup(duration=af.duration)
```

`source` accepts a parquet/csv path or an in-memory DataFrame. `dimensions` maps each lookup key to either a column name in the source (`"duration": "duration"`) or a `Dimension` object that transforms or computes the key. `value` names the column the lookup returns.

The lookup is a single call. It works on scalar columns, list columns (returning aligned vectors per period), and Polars expressions.

## Wide and Tidy

Most actuarial tables come from Excel, regulator publications, or vendor systems in **wide** shape — ages down the side, sex/smoker codes or durations across the top:

```text
┌─────┬───────┬───────┬───────┬───────┐
│ age │  MNS  │  FNS  │  MS   │  FS   │
├─────┼───────┼───────┼───────┼───────┤
│ 30  │0.0011 │0.0010 │0.0021 │0.0019 │
│ 31  │0.0012 │0.0011 │0.0022 │0.0020 │
└─────┴───────┴───────┴───────┴───────┘
```

`Table` loads either shape. For wide tables, declare a `MeltDimension` for the axis sitting across the columns and the wide → tidy reshape happens once, at registration:

```python
from gaspatchio.assumptions import Table, MeltDimension

# A wide mortality table as received: age down the side, sex/smoker codes
# (Male Non-Smoker, Female Non-Smoker, Male Smoker, Female Smoker) across the top.
wide_df = pl.DataFrame(
    {
        "age": [30, 31, 32, 33, 34],
        "MNS": [0.0011, 0.0012, 0.0013, 0.0014, 0.0015],
        "FNS": [0.0010, 0.0011, 0.0012, 0.0013, 0.0014],
        "MS": [0.0021, 0.0022, 0.0023, 0.0024, 0.0025],
        "FS": [0.0019, 0.0020, 0.0021, 0.0022, 0.0023],
    }
)

mortality = Table(
    name="mortality_rates",
    source=wide_df,
    dimensions={
        "age": "age",
        "sex_smoker": MeltDimension(
            columns=["MNS", "FNS", "MS", "FS"],
            name="sex_smoker",
        ),
    },
    value="mortality_rate",
)

af = ActuarialFrame(
    pl.DataFrame(
        {
            "policy_id": ["P001", "P002"],
            "age": [30, 32],
            "sex_smoker": ["MNS", "FS"],
        }
    )
)
af.mort_rate = mortality.lookup(age=af.age, sex_smoker=af.sex_smoker)
```

Lookups are always against the tidy form. Keep your source files in whatever shape you receive them — the reshape is one-off, at load time.

## Ultimate Rates and Overflow

Select-ultimate tables have explicit columns for the first N durations (typically 5, 15, or 25) and one "Ult." column carrying the rate beyond the select period. `ExtendOverflow` expands that column into explicit durations at registration, so a lookup at duration 50 hits a real row rather than missing or falling through:

```python
from gaspatchio.assumptions import (
    Table,
    MeltDimension,
    ExtendOverflow,
    DataDimension,
)

# The 2015 VBT as published: issue age down the side, durations 1–25 plus an
# "Ultimate" column across the top. Drop the derived attained-age column.
vbt_df = pl.read_csv("2015-VBT-FSM-ANB.csv").drop("Attained Age")

vbt = Table(
    name="vbt_2015",
    source=vbt_df,
    dimensions={
        "issue_age": DataDimension(column="Issue Age", rename_to="issue_age"),
        "duration": MeltDimension(
            columns=[str(i) for i in range(1, 26)] + ["Ultimate"],
            name="duration",
            overflow=ExtendOverflow("Ultimate", to_value=120),
        ),
    },
    value="qx",
)
```

Durations 26 through 120 each become real rows carrying the Ultimate rate. Per-policy projections that step past the select horizon look up normally, without `duration.clip(upper_bound=25)` boilerplate in your model code.

For sex/smoker × age × duration mortality with all of these conventions in one place — plus age-basis validation and automatic select-period clamping — reach for [MortalityTable](https://gaspatchio.dev/0.9.0/concepts/mortality/index.md). It wraps `Table` and adds the structure-aware checks on top.

## Vector Lookups Across the Projection

The actuary's natural unit is the **vector** — a rate per period, per policy. `lookup()` works on list-columns and returns aligned vectors:

```python
# A select-ultimate mortality table keyed on age × sex/smoker × duration.
mort_rows = [
    {
        "age": age,
        "sex_smoker": ss,
        "duration": dur,
        "mortality_rate": round(0.0010 * (1 + 0.05 * dur) * (1.8 if ss in ("MS", "FS") else 1.0), 6),
    }
    for age in range(30, 71)
    for ss in ["MNS", "FNS", "MS", "FS"]
    for dur in range(0, 6)
]
mortality = Table(
    name="mortality_select",
    source=pl.DataFrame(mort_rows),
    dimensions={"age": "age", "sex_smoker": "sex_smoker", "duration": "duration"},
    value="mortality_rate",
)

# Two policies, each with a five-period projection (months 0, 12, 24, 36, 48).
af = ActuarialFrame(
    pl.DataFrame(
        {
            "policy_id": ["P001", "P002"],
            "issue_age": [30, 35],
            "month": [[0, 12, 24, 36, 48], [0, 12, 24, 36, 48]],
            "gender": ["M", "F"],
            "smoker": ["NS", "S"],
        }
    )
)

# Per-policy projection vectors
af.attained_age = af.issue_age + af.month // 12     # list[i64] per policy
af.duration = af.month // 12                          # list[i64] per policy
af.sex_smoker = af.gender + af.smoker                 # scalar per policy

# Single call returns a vector of rates aligned to the input vectors
af.mort_rate = mortality.lookup(
    age=af.attained_age,
    sex_smoker=af.sex_smoker,
    duration=af.duration,
)
```

No exploding model points into one row per period, no per-period joining, no aggregation back. The lookup matches the actuary's mental model: one rate vector per policy, the same length as the projection.

## When to Use What

| Need                                                                         | Reach for                                                                                                   |
| ---------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------- |
| Tidy source with one or more dimension columns                               | `Table(..., dimensions={"age": "age"}, value="qx")`                                                         |
| Wide source with codes across columns (MNS/FNS, age bands, scenario ids)     | `Table(..., dimensions={"code": MeltDimension(columns=[...])})`                                             |
| Select-then-ultimate with an "Ult." column                                   | `MeltDimension(..., overflow=ExtendOverflow("Ult.", to_value=120))`                                         |
| Dimension derived from other columns (`attained_age = issue_age + duration`) | `ComputedDimension(...)`                                                                                    |
| Constant categorical (e.g. `table_id` stamped at registration)               | `CategoricalDimension(...)`                                                                                 |
| Multi-step construction or programmatic configuration                        | `TableBuilder(...)` — see the [API reference](https://gaspatchio.dev/0.9.0/api/assumptions/index.md)        |
| Mortality with age basis + select/ultimate + joint conventions               | [`MortalityTable`](https://gaspatchio.dev/0.9.0/concepts/mortality/index.md) on top of `Table`              |
| Economic curves with key-rate sensitivities                                  | [`Curve`](https://gaspatchio.dev/0.9.0/concepts/curves/index.md) — a separate object, not a `Table` wrapper |

## Governance: Table Identity

`source_sha()` pins down the registered table — its rows, schema, dimension configuration, and overflow strategy — as a single sha256:

```python
mortality.source_sha()
# 'sha256:e4ef39b8be6b3c75f6b6f2a0d7e5e2c1b8a9d6f7e5c4b3a2f1e0d9c8b7a6f5e4'
```

Record this alongside each valuation. Two tables with the same source data and the same dimensions have the same `source_sha()`; reload from a different file, swap a `MeltDimension` for an `ExtendOverflow`, or drop a row and the value changes — so any drift between quarters is visible the moment you compare runs. The Table's identity rolls up into the rollforward's [overall version stamp](https://gaspatchio.dev/0.9.0/concepts/rollforward/inspection/index.md), so an assumption-table change between valuations shows up at the model level without separate tracking.

## Runnable Companions

The patterns above run end-to-end in the mini-VA tutorial:

- `bindings/python/gaspatchio/tutorials/level-3-mini-va-typed/steps/01-from-files/model.py` — mortality and lapse tables loaded from parquet, vector lookups against per-policy projection vectors
- `bindings/python/gaspatchio/tutorials/level-3-mini-va-typed/steps/02-select-mort/model.py` — `ExtendOverflow` + sex-based `table_id` lookup as a third dimension

## See Also

- [Mortality Tables](https://gaspatchio.dev/0.9.0/concepts/mortality/index.md) — `MortalityTable` wraps `Table` and adds age-basis + select/ultimate conventions
- [Curves](https://gaspatchio.dev/0.9.0/concepts/curves/index.md) — `Curve` carries the discount-rate term structure as a separate object
- [Worked Examples](https://gaspatchio.dev/0.9.0/concepts/assumptions_examples/index.md) — end-to-end model integrating mortality, lapse, and premium tables
- [Scenarios → Table Sensitivities](https://gaspatchio.dev/0.9.0/concepts/scenarios/table-sensitivities/index.md) — shocking assumption tables across scenarios
- [API: Assumptions](https://gaspatchio.dev/0.9.0/api/assumptions/index.md) — full surface: `Table`, `TableBuilder`, all Dimension and Strategy types

# Assumption Table Examples in Gaspatchio

## Working with Mortality Tables

This guide walks through using the 2015 VBT Female Smoker Mortality Table (ANB) as an example to demonstrate how to set up and use assumption tables in Gaspatchio.

### Understanding the Table Structure

The 2015 VBT table is structured as follows:

- Rows represent issue ages (18-95)
- Columns represent policy durations (1-25 plus "Ultimate")
- Values represent mortality rates per 1,000

Here's a small sample from the table:

| Issue Age | Duration 1 | Duration 2 | Duration 3 | Duration 4 | Duration 5 | Ultimate | Attained Age |
| --------- | ---------- | ---------- | ---------- | ---------- | ---------- | -------- | ------------ |
| 30        | 0.20       | 0.25       | 0.31       | 0.38       | 0.45       | 4.84     | 55           |
| 31        | 0.21       | 0.26       | 0.34       | 0.42       | 0.51       | 5.35     | 56           |
| 32        | 0.22       | 0.28       | 0.37       | 0.47       | 0.58       | 5.93     | 57           |
| 33        | 0.23       | 0.31       | 0.42       | 0.53       | 0.65       | 6.59     | 58           |
| 34        | 0.25       | 0.35       | 0.48       | 0.61       | 0.73       | 7.31     | 59           |

### Loading the Assumption Table

Loading assumption tables is straightforward with the dimension-based API. Gaspatchio provides tools to analyze table structure and configure dimensions:

```python
import polars as pl
import gaspatchio as gs

# Load the select-and-ultimate columns only (drop the derived Attained Age column)
df = pl.read_csv("2015-VBT-FSM-ANB.csv").drop("Attained Age")

# Analyze table structure (optional but helpful)
schema = gs.assumptions.analyze_table(df)
print(schema.suggest_table_config())

# Load the mortality table with dimension configuration
vbt_table = gs.Table(
    name="vbt_2015_female_smoker",
    source=df,
    dimensions={
        "issue_age": gs.assumptions.DataDimension(
            column="Issue Age", rename_to="issue_age"
        ),
        "duration": gs.assumptions.MeltDimension(
            columns=[str(i) for i in range(1, 26)] + ["Ultimate"],
            name="duration",
            overflow=gs.assumptions.ExtendOverflow("Ultimate", to_value=200)
        )
    },
    value="mortality_rate"
)
```

The API explicitly configures:

- **Dimension mapping**: `DataDimension(column="Issue Age", rename_to="issue_age")` renames the source column to the actuarial dimension name you use in `lookup(issue_age=...)`. Shorthand `"col_name"` works when the lookup name and the source column name are identical; use the full form when they differ.
- **Melt dimensions**: Transform wide columns (1-25, Ultimate) into long format
- **Overflow strategies**: Expand "Ultimate" values to higher durations
- **Value column name**: Name for the melted rates

After loading, the internal data looks like this:

| issue_age | duration | mortality_rate |
| --------- | -------- | -------------- |
| 30        | 1        | 0.20           |
| 30        | 2        | 0.25           |
| 30        | 3        | 0.31           |
| 30        | 4        | 0.38           |
| 30        | 5        | 0.45           |
| 30        | 26       | 4.84           |
| 30        | 27       | 4.84           |
| 30        | 150      | 4.84           |
| ...       | ...      | ...            |

### Using the Assumption Table in ActuarialFrame

Now we can use this table for lightning-fast lookups.

Why So Fast?

This VBT table has dimensions `[78 ages × 200 durations]` = 15,600 entries. Gaspatchio detects this as a **dense table** and stores it as a contiguous array in memory. Each lookup is just:

1. Compute index: `age_offset × 200 + duration` (a few nanoseconds)
1. Read value: `data[index]` (direct memory access)

No hash computation, no bucket probing - just arithmetic and array indexing. This is why 324 million lookups complete in ~1 second instead of ~27 seconds.

```python
# Create a simple policy dataset
policy_data = pl.DataFrame({
    "policy_id": ["A001", "A002", "A003", "A004"],
    "issue_age": [30, 35, 40, 45],
    "duration": [1, 3, 5, 10]
})

# Convert to ActuarialFrame
af = gs.ActuarialFrame(policy_data)

# Look up mortality rates using the table's lookup method
af.mortality_rate = vbt_table.lookup(issue_age=af.issue_age, duration=af.duration)

print(af)
```

Result:

```text
shape: (4, 4)
┌──────────┬───────────┬──────────┬───────────────┐
│ policy_id ┆ issue_age ┆ duration ┆ mortality_rate │
│ ---      ┆ ---       ┆ ---      ┆ ---           │
│ str      ┆ i64       ┆ i64      ┆ f64           │
╞══════════╪═══════════╪══════════╪═══════════════╡
│ A001     ┆ 30        ┆ 1        ┆ 0.20          │
│ A002     ┆ 35        ┆ 3        ┆ 0.54          │
│ A003     ┆ 40        ┆ 5        ┆ 1.15          │
│ A004     ┆ 45        ┆ 10       ┆ 4.10          │
└──────────┴───────────┴──────────┴───────────────┘
```

### Working with Overflow Durations

The beauty of the API is that overflow handling is completely transparent. Even extreme durations work instantly:

```python
# Test with durations beyond the table (> 25)
extreme_data = pl.DataFrame({
    "policy_id": ["X001", "X002"],
    "issue_age": [30, 40], 
    "duration": [50, 100]  # Way beyond table max of 25!
})

af_extreme = gs.ActuarialFrame(extreme_data)
af_extreme.mortality_rate = vbt_table.lookup(
    issue_age=af_extreme.issue_age,
    duration=af_extreme.duration
)

print(af_extreme)
```

Result:

```text
shape: (2, 4)
┌──────────┬───────────┬──────────┬────────────────┐
│ policy_id ┆ issue_age ┆ duration ┆ mortality_rate │
│ ---      ┆ ---       ┆ ---      ┆ ---            │
│ str      ┆ i64       ┆ i64      ┆ f64            │
╞══════════╪═══════════╪══════════╪════════════════╡
│ X001     ┆ 30        ┆ 50       ┆ 4.84           │
│ X002     ┆ 40        ┆ 100      ┆ 9.32           │
└──────────┴───────────┴──────────┴────────────────┘
```

Both policies get the "Ultimate" rate because the `ExtendOverflow` strategy pre-expanded the overflow during loading.

### Projecting Multiple Periods

Gaspatchio's vector-based approach works seamlessly with the API:

```python
# Create a policy with projection over multiple durations
policy_projection = pl.DataFrame({
    "policy_id": ["B001"],
    "issue_age": [30],
    "duration": [[1, 2, 3, 4, 5, 25, 26, 50, 100]]  # Mix of regular and overflow
})

af_proj = gs.ActuarialFrame(policy_projection)

# Look up mortality rates for all durations at once
af_proj.mortality_rate = vbt_table.lookup(
    issue_age=af_proj.issue_age,
    duration=af_proj.duration
)

# Explode the per-policy vectors to one row per duration for visualization
result = af_proj.collect().explode(["duration", "mortality_rate"])
print(result)
```

Result:

```text
shape: (9, 4)
┌──────────┬───────────┬──────────┬───────────────┐
│ policy_id ┆ issue_age ┆ duration ┆ mortality_rate │
│ ---      ┆ ---       ┆ ---      ┆ ---           │
│ str      ┆ i64       ┆ i64      ┆ f64           │
╞══════════╪═══════════╪══════════╪═══════════════╡
│ B001     ┆ 30        ┆ 1        ┆ 0.20          │
│ B001     ┆ 30        ┆ 2        ┆ 0.25          │
│ B001     ┆ 30        ┆ 3        ┆ 0.31          │
│ B001     ┆ 30        ┆ 4        ┆ 0.38          │
│ B001     ┆ 30        ┆ 5        ┆ 0.45          │
│ B001     ┆ 30        ┆ 25       ┆ 4.12          │
│ B001     ┆ 30        ┆ 26       ┆ 4.84          │
│ B001     ┆ 30        ┆ 50       ┆ 4.84          │
│ B001     ┆ 30        ┆ 100      ┆ 4.84          │
└──────────┴───────────┴──────────┴───────────────┘
```

### Loading Simple Curves

For 1-dimensional tables (like lapse rates by age), the API is even simpler:

```python
# Load a simple age → lapse rate curve
lapse_table = gs.Table(
    name="lapse_2025",
    source="lapse_curve.csv",
    dimensions={
        "age": "age"  # Simple string shorthand
    },
    value="lapse_rate"
)

# Use it immediately against a frame carrying attained age
af_lapse = gs.ActuarialFrame(
    pl.DataFrame({"policy_id": ["A001", "A002"], "age": [30, 45]})
)
af_lapse.lapse_rate = lapse_table.lookup(age=af_lapse.age)
print(af_lapse)
```

### Advanced Features

For more complex scenarios, you have full control with the dimension-based API:

```python
# Multi-dimensional table with selective column loading
mortality_table = gs.Table(
    name="mortality_by_gender",
    source="mortality_m_f.csv",
    dimensions={
        "age": "age",
        "gender": gs.assumptions.MeltDimension(
            columns=["Male", "Female"],
            name="gender"
        )
    },
    value="mortality_rate"
)

# Table with custom overflow limits
salary_table = gs.Table(
    name="salary_scale",
    source="salary_by_service.csv",
    dimensions={
        "grade": "grade",
        "service": gs.assumptions.MeltDimension(
            columns=[str(i) for i in range(1, 21)] + ["20+"],
            name="service",
            overflow=gs.assumptions.ExtendOverflow("20+", to_value=50)
        )
    },
    value="scale_factor"
)

# A tidy source keyed on issue age and policy year, with an attained-age
# dimension derived at registration.
df = pl.DataFrame({
    "issue_age": [30, 30, 40, 40],
    "policy_year": [1, 2, 1, 2],
    "assumption_value": [0.010, 0.012, 0.020, 0.024],
})

# Using computed dimensions
complex_table = gs.Table(
    name="complex_assumptions",
    source=df,
    dimensions={
        "issue_age": "issue_age",
        "policy_year": "policy_year",
        "attained_age": gs.assumptions.ComputedDimension(
            pl.col("issue_age") + pl.col("policy_year") - 1,
            "attained_age"
        )
    },
    value="assumption_value"
)
```

### Using the TableBuilder Pattern

For step-by-step table construction, use the fluent `TableBuilder` API:

```python
# Build a complex mortality table
mortality_table = (
    gs.TableBuilder("mortality_select_ultimate")
    .from_source("mortality_su.csv")
    .with_data_dimension("issue_age", "IssueAge", rename_to="issue_age")
    .with_data_dimension("gender", "Gender", rename_to="gender")
    .with_melt_dimension(
        "duration",
        columns=[f"Dur{i}" for i in range(1, 16)] + ["Ultimate"],
        overflow=gs.assumptions.ExtendOverflow("Ultimate", to_value=100),
        fill=gs.assumptions.LinearInterpolate()  # Interpolate any gaps
    )
    .with_value_column("qx_rate")
    .build()
)

# The table is ready for lookups
af_su = gs.ActuarialFrame(
    pl.DataFrame({
        "policy_id": ["C001", "C002"],
        "age": [30, 31],
        "sex": ["Male", "Female"],
        "policy_duration": [3, 10],
    })
)
af_su.mortality_rate = mortality_table.lookup(
    issue_age=af_su.age,
    gender=af_su.sex,
    duration=af_su.policy_duration
)
print(af_su)
```

### Metadata and Table Discovery

Tables can include metadata for documentation and discovery:

```python
# The wide VBT source carries durations 1–25 plus an "Ultimate" column.
duration_columns = [str(i) for i in range(1, 26)] + ["Ultimate"]

# Create table with rich metadata
vbt_table = gs.Table(
    name="vbt_2015_complete",
    source="vbt_2015_all.csv",
    dimensions={
        "age": "Age",
        "gender": "Gender",
        "smoking": "Smoker",
        "duration": gs.assumptions.MeltDimension(
            columns=duration_columns,
            name="duration",
            overflow=gs.assumptions.ExtendOverflow("Ultimate", to_value=120)
        )
    },
    value="mortality_rate",
    metadata={
        "source": "2015 Valuation Basic Table",
        "basis": "ANB",
        "version": "2015",
        "effective_date": "2015-01-01",
        "description": "Industry standard mortality table",
        "tags": ["mortality", "vbt", "2015", "standard"]
    }
)

# Discover tables
all_tables = gs.list_tables()
print(f"Available tables: {all_tables}")

# Get metadata for a specific table
metadata = gs.get_table_metadata("vbt_2015_complete")
print(f"Table metadata: {metadata}")

# List all tables with metadata
tables_info = gs.list_tables_with_metadata()
for name, meta in tables_info.items():
    print(f"{name}: {meta.get('description', 'No description')}")
```

# Curves

## Discount Rates Belong in Their Own Object

Present-value calculations run against a curve — zero rates from a regulator, a swap curve from the market, a stress shifted off a baseline. A single flat rate stuffed into a column works for a first cut; a `Curve` carries a full term structure, supports parallel and key-rate stresses, and can be reconciled tenor-by-tenor against a vendor's discounting.

A `Curve` holds the discount-rate term structure as a single named value. You build it once from zero rates or par rates, and read off spot rates, discount factors, and forward rates by tenor. Sensitivity stresses produce new Curves you can swap into the same pricing logic with no code changes — only the curve differs.

## Building a Curve

The two construction paths cover most actuarial use cases.

### `from_zero_rates` — most common

You already have continuously- or annually-compounded zero rates at a list of tenors (often a regulator-published set). Build the Curve directly:

```python
from gaspatchio import Curve

curve = Curve.from_zero_rates(
    tenors=[0.5, 1.0, 2.0, 5.0, 10.0, 20.0, 30.0],
    rates=[0.04, 0.041, 0.042, 0.044, 0.046, 0.045, 0.044],
)
```

Tenors are in years. Rates are annually compounded. Between knot tenors the curve interpolates linearly on the rate by default — see [Interpolation and parametric curves](#interpolation-and-parametric-curves) for log-linear, monotone-cubic, and parametric (Svensson, Smith-Wilson) alternatives.

### `from_par_rates` — when you start from market quotes

You have par yields at standard tenors (the typical market quote shape). The Curve bootstraps zero rates from the par rates internally:

```python
curve = Curve.from_par_rates(
    tenors=[1.0, 2.0, 3.0, 4.0, 5.0],
    par_rates=[0.041, 0.042, 0.043, 0.0435, 0.044],
)
```

`from_par_rates` requires consecutive annual tenors starting at 1 — the bootstrap walks the curve year-by-year, so the input has to be dense. Use it when reconciling against a vendor that quotes par yields at every year; use `from_zero_rates` when the input is already a zero curve from a regulator publication or model output.

## Interpolation and Parametric Curves

A discount curve is only pinned down at its knot tenors; everything between and beyond them is a modelling choice. The default — a straight line on the zero rates — is transparent and reconciles cleanly, but two situations call for more: a smoother shape between liquid points, and a principled extrapolation past the last traded tenor to a regulator-specified ultimate rate.

### Choosing an interpolation

`from_zero_rates` and `from_par_rates` take an `interpolation=` argument. Every method passes through the input knots exactly; they differ only between knots. What happens *past* the grid is a separate choice — see [Extrapolation: beyond the last knot](#extrapolation-beyond-the-last-knot).

| `interpolation=`     | Between knots                                                                           | Reach for it when                                                                 |
| -------------------- | --------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------- |
| `"linear"` (default) | Straight line on the zero rates                                                         | The regulator publishes a rate at every tenor you need; transparency matters most |
| `"log_linear"`       | Straight line in log-discount-factor space — a piecewise-constant forward rate          | You discount on a money-market basis, or want constant forwards between knots     |
| `"pchip"`            | Monotone cubic (Fritsch-Carlson) — smooth, but shape-preserving, so it never overshoots | You need a smooth curve and a plain cubic spline would introduce spurious humps   |

```python
from gaspatchio import Curve

knots = {"tenors": [1.0, 2.0, 5.0, 10.0], "rates": [0.020, 0.025, 0.030, 0.033]}
linear_curve = Curve.from_zero_rates(**knots)                              # default
log_linear_curve = Curve.from_zero_rates(**knots, interpolation="log_linear")
monotone_curve = Curve.from_zero_rates(**knots, interpolation="pchip")
```

### Extrapolation: beyond the last knot

A pension or whole-of-life projection discounts cashflows at 30, 40, 50 years; liquid market quotes often stop at 10 or 20. Everything past the last knot is convention, not observation — and the convention you pick moves long-tail present values by real money. So it is an explicit argument on `from_zero_rates` and `from_par_rates`, not a side effect of the interpolation method:

| `extrapolation=`                | Beyond the last knot                                                                                        | Reach for it when                                                                           |
| ------------------------------- | ----------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------- |
| `"flat"` (default)              | Holds the boundary **spot rate** — the 30y read equals the last-knot rate                                   | Transparency and reconcilability matter most; the tail is easy to explain and easy to check |
| `"forward"` (`log_linear` only) | Holds the last segment's **forward rate** — the curve keeps compounding at the rate the market last implied | Market-consistent discounting of cashflows well beyond the last liquid tenor                |

```python
from gaspatchio import Curve

knots = {"tenors": [1.0, 2.0, 5.0, 10.0], "rates": [0.020, 0.025, 0.030, 0.033]}
flat_tail = Curve.from_zero_rates(**knots, interpolation="log_linear")
forward_tail = Curve.from_zero_rates(
    **knots, interpolation="log_linear", extrapolation="forward"
)

print(f"5y->10y forward:   {flat_tail.forward_rate(t1=5.0, t2=10.0):.4%}")
print(f"30y spot, flat:    {flat_tail.spot_rate(30.0):.4%}")
print(f"30y spot, forward: {forward_tail.spot_rate(30.0):.4%}")
print(f"30y DF, flat:      {flat_tail.discount_factor(30.0):.6f}")
print(f"30y DF, forward:   {forward_tail.discount_factor(30.0):.6f}")
```

```text
5y->10y forward:   3.6009%
30y spot, flat:    3.3000%
30y spot, forward: 3.5005%
30y DF, flat:      0.377564
30y DF, forward:   0.356228
```

Inside the knot grid the two curves are identical — the choice only shows up past the boundary. That is exactly why it deserves an explicit argument: a tail convention that never appears in a short-dated reconciliation still moves a 40-year liability.

Both options clamp the **rate**, at both ends of the grid. Clamping the discount factor instead — holding the log-discount-factor level past the boundary — implies a spot rate that decays toward zero: on the curve above it reads 1.09% at 30y and prices a dollar due in 30 years as if it were due in 10. Below the first knot the same clamp errs the other way, reading 4.04% at six months against a 2% one-year rate. Gaspatchio holds rates, not discount factors, so the tail stays at the boundary rate rather than drifting to either extreme.

`"forward"` needs a forward rate to hold, and only the log-discount-factor representation defines one — on `"linear"` and `"pchip"` knots, `extrapolation="forward"` raises `ValueError` at construction rather than silently behaving like `"flat"`.

For a regulator-mandated convergence to an ultimate forward rate — the Solvency II shape — extrapolation is not a clamp at all: build the curve with [Smith-Wilson](#parametric-curves-svensson-and-smith-wilson) below. The parametric constructors take no `extrapolation=` argument, because the formula itself defines the tail.

### Parametric curves: Svensson and Smith-Wilson

Where interpolation reads *between* the knots you supply, a parametric curve is defined by a *formula* — the same form central banks publish, or the extrapolation regulators mandate beyond the last liquid tenor.

**Nelson-Siegel-Svensson (NSS)** is the six-parameter form the Federal Reserve and ECB publish for the nominal yield curve — a level, a slope, and two curvature humps with their own decay rates. Build one from published parameters, or calibrate the parameters to market knots:

```python
nss = Curve.from_svensson(b0=0.04, b1=-0.01, b2=0.005, b3=0.002, tau1=1.5, tau2=10.0)
nss_fitted = Curve.fit_svensson(
    tenors=[1.0, 2.0, 5.0, 10.0, 20.0, 30.0],   # six observations minimum (six NSS parameters)
    rates=[0.030, 0.032, 0.035, 0.037, 0.038, 0.039],
)
```

**Smith-Wilson** is the extrapolation method behind EIOPA's risk-free interest-rate term structures under Solvency II. It reproduces the liquid market rates exactly, then pulls the curve smoothly toward an Ultimate Forward Rate (UFR) beyond the last liquid point:

```python
sw = Curve.fit_smith_wilson(
    tenors=[1.0, 2.0, 4.0, 5.0, 6.0, 7.0],
    rates=[0.01, 0.02, 0.03, 0.032, 0.035, 0.04],
    ufr=0.04,        # ultimate forward rate the curve converges to
    alpha=0.15,      # convergence speed; omit (alpha=None) to auto-calibrate to the EIOPA floor
)

sw_20y = sw.spot_rate(20.0)   # extrapolated well beyond the 7y last-liquid-point, toward the UFR
```

Both constructors return an ordinary `Curve` — `spot_rate`, `discount_factor`, the stresses, and `source_sha()` all behave identically, so a parametric curve drops into the same pricing and governance flow as a knot-based one.

## Reading Off the Curve

```python
curve.spot_rate(t=5.0)            # zero rate at 5y, scalar
curve.discount_factor(t=10.0)     # PV(1 received at t=10), scalar
curve.forward_rate(t1=5.0, t2=10.0)   # 5y forward, 5y tenor
```

`spot_rate` and `discount_factor` also accept a Polars Series, Polars expression, list, or `af["col"]` reference — so you can feed Schedule-derived `t` values straight through:

```python
from datetime import date
from gaspatchio import Schedule

sched = Schedule.from_calendar_grid(
    start_date=date(2025, 1, 31), n_periods=240, frequency="1M"
)
# n_periods=240 produces 241 cumulative year-fractions (one per period
# boundary, inclusive); drop the leading 0 so the vector aligns with the
# 240 period-end cashflows.
t_years = sched.cumulative_year_fractions()[1:]    # 240 floats
disc_factors = curve.discount_factor(t=t_years)    # 240 floats
```

`discount_factor` accepts a Python list, a Polars Series or expression, an `af["col"]` reference — anything that resolves to scalar year-fractions. To broadcast the resulting vector onto every policy, assign it as a list column:

```python
import polars as pl
from gaspatchio import ActuarialFrame

policies = pl.DataFrame({"premium": [[100.0] * 240, [200.0] * 240]})
policies = policies.with_columns(
    pl.Series("df", [disc_factors] * len(policies), dtype=pl.List(pl.Float64))
)
af = ActuarialFrame(policies)
af.pv_premium = (af["premium"] * af["df"]).list.sum()
```

## Sensitivity Stresses

`Curve` supports parallel and key-rate sensitivities directly. Both are pure-function transforms — they return a new Curve, leaving the original untouched, so you can run the same pricing logic against any combination of them without restating your baseline.

### Parallel shift

Shift every zero rate by the same number of basis points:

```python
up_100 = curve.shift_parallel(bps=100)
down_50 = curve.shift_parallel(bps=-50)
```

Run the same pricing logic against `curve`, `up_100`, and `down_50` to compute DV01-style sensitivities.

### Key-rate shift

Shift exactly one knot tenor by a basis-point amount, leaving all other knots unchanged. Used for key-rate-duration analysis and for isolating the sensitivity to a single point on the curve:

```python
key_5y = curve.key_rate_shift(tenor=5.0, bps=50)
```

The `tenor=` argument must be one of the knot tenors used to build the curve. Shifting at a non-knot tenor raises `ValueError` — the curve doesn't infer between-knot perturbation rules.

## Worked Example

A flat 4% curve, three readings, and two stresses:

```python
from gaspatchio import Curve

flat = Curve.from_zero_rates(
    tenors=[0.5, 1.0, 2.0, 5.0, 10.0, 20.0, 30.0],
    rates=[0.04] * 7,
)

print(f"Spot 1y:  {flat.spot_rate(1.0):.6f}")
print(f"Spot 5y:  {flat.spot_rate(5.0):.6f}")
print(f"Spot 7y:  {flat.spot_rate(7.0):.6f}")
print(f"DF 1y:    {flat.discount_factor(1.0):.6f}")
print(f"DF 5y:    {flat.discount_factor(5.0):.6f}")
print(f"DF 10y:   {flat.discount_factor(10.0):.6f}")
print(f"5y->10y forward: {flat.forward_rate(t1=5.0, t2=10.0):.6f}")

parallel = flat.shift_parallel(bps=100)
print(f"After +100bp parallel — DF 5y: {parallel.discount_factor(5.0):.6f}")

key = flat.key_rate_shift(tenor=5.0, bps=50)
print(f"After +50bp at 5y    — DF 5y: {key.discount_factor(5.0):.6f}")
print(f"                       DF 1y: {key.discount_factor(1.0):.6f}")
print(f"                       DF 10y: {key.discount_factor(10.0):.6f}")
```

```text
Spot 1y:  0.040000
Spot 5y:  0.040000
Spot 7y:  0.040000
DF 1y:    0.961538
DF 5y:    0.821927
DF 10y:   0.675564
5y->10y forward: 0.040000
After +100bp parallel — DF 5y: 0.783526
After +50bp at 5y    — DF 5y: 0.802451
                       DF 1y: 0.961538
                       DF 10y: 0.675564
```

The parallel shift moves every discount factor; the key-rate shift moves only the 5y point and lets nearby tenors absorb the perturbation through interpolation. The 1y and 10y discount factors after the key-rate shift sit between the unstressed and parallel-stressed values — exactly what a single-knot sensitivity should produce.

## Governance: Curve Identity

Quarter-over-quarter curve drift is one of the easiest contributors to miss in a movement analysis. A regulator publishes a fresh zero curve, someone rebuilds with a slightly different tenor set, a stale copy slips back in from an old notebook — and the AOM ends up with discounting movement nobody set out to put there. `source_sha()` gives the curve a single value that captures its construction (the tenors, the rates, the day count, the interpolation and extrapolation choices, the parametric form if any) so a quiet drift can't go unnoticed:

```python
flat.source_sha()
# 'sha256:a594be4f4ede393146f83f32f895645c56118a0140af855da001761f84f8e890'
```

Record this alongside each valuation. Two curves with the same tenors and rates have the same `source_sha()`; a regulator update that moves a single rate by a basis point gives you a different one. The Curve's identity also rolls up into the rollforward's [overall version stamp](https://gaspatchio.dev/0.9.0/concepts/rollforward/inspection/index.md), so a curve change between quarters is visible at the model level without separate tracking — and a curve change you didn't expect to see is one you can spot before it shows up on the AOM.

## Runnable Companions

The patterns above run end-to-end in the mini-VA tutorial:

- `bindings/python/gaspatchio/tutorials/level-3-mini-va-typed/base/model.py` — flat 4% curve, discount factors fed from cumulative year fractions
- `bindings/python/gaspatchio/tutorials/level-3-mini-va-typed/steps/05-rate-curves/model.py` — non-flat curve plus `shift_parallel(bps=100)` and `key_rate_shift(tenor=5.0, bps=50)` worked through
- `bindings/python/gaspatchio/tutorials/patterns/curves-and-scheduling/05_interpolation_methods.py` — the three interpolations plus the Svensson and Smith-Wilson parametric curves, each asserted against a closed form or the published lifelib spot values

## See Also

- [Schedules](https://gaspatchio.dev/0.9.0/concepts/schedules/index.md) — `cumulative_year_fractions()` is the natural input to `discount_factor(t=...)`
- [Inspection](https://gaspatchio.dev/0.9.0/concepts/rollforward/inspection/index.md) — the Curve's identity rolls up into the rollforward's overall version stamp

# Intro

Gaspatchio is a Python actuarial modelling framework. You build your projection vectors — survival, account values, cashflows, reserves — as columns on an `ActuarialFrame`. The whole projection runs across every policy together, and every output traces back to the inputs and steps that produced it.

## The unit of work: `ActuarialFrame`

`ActuarialFrame` is what the model is built on. Column assignments are recorded, not executed, until you call `.collect()` — at which point the whole calculation runs as a single plan. Column lineage is tracked through that plan, so every output column traces back to the inputs and intermediate steps that produced it.

```python
import datetime
from gaspatchio import ActuarialFrame

af = ActuarialFrame({
    "policy_id": ["P001", "P002"],
    "age": [35, 42],
    "issue_date": [datetime.date(2022, 1, 15), datetime.date(2021, 6, 1)],
    "valuation_date": [datetime.date(2024, 12, 31), datetime.date(2024, 12, 31)],
})

af.days_in_force = af.valuation_date.excel.days(af.issue_date)
af.years_in_force = af.issue_date.excel.yearfrac(af.valuation_date, basis="act/act").round(2)

df = af.collect()
```

## How an actuary works in it

Four moves cover almost every model:

1. **Set the time axis.** Declare the projection — how many periods, what frequency, which calendar, which day-count. Anniversary masks, year-fractions, and period dates all derive from it. See [Schedules](https://gaspatchio.dev/0.9.0/concepts/schedules/index.md).
1. **Register assumptions.** Mortality, lapse, expenses, economic curves. Tables sit in an in-memory registry with composite-key lookups that work on scalar columns and on list columns at projection time. See [Assumptions](https://gaspatchio.dev/0.9.0/concepts/assumptions/index.md).
1. **Write the projection.** Express the calculation as column assignments and time-series operations. For pure cumulative accumulation (survival probabilities, discount factors, AV with pre-computable charges), use the cumulative ops on the [Projections](https://gaspatchio.dev/0.9.0/concepts/projections/index.md) page. For state-dependent calculations (COI on net amount at risk, IUL floor/cap, GMDB ratchet), use [Rollforward](https://gaspatchio.dev/0.9.0/concepts/rollforward/index.md).
1. **Stress and run.** Reuse the same calculation across shocked assumption tables and scenario configs to produce CTE, quantile, and other risk measures. See [Scenarios](https://gaspatchio.dev/0.9.0/concepts/scenarios/index.md).

## What's next

Start with [Schedules](https://gaspatchio.dev/0.9.0/concepts/schedules/index.md) — every projection lives on a time axis, and getting that right is what everything else hangs off. From there, [Projections](https://gaspatchio.dev/0.9.0/concepts/projections/index.md) walks the closed-form cumulative pattern through to the linear recurrence (`accumulate()`) and the handover to [Rollforward](https://gaspatchio.dev/0.9.0/concepts/rollforward/index.md) for state-dependent products.

# Mortality Tables

## Mortality Has Conventions Worth Encoding

Mortality lookup is more than "row by age". The age you look up at depends on whether the table is quoted on age last birthday, age nearest birthday, or age next birthday. Many product tables are select-ultimate, so the lookup also needs a duration — and the duration must clamp to the select period beyond which all rows roll into the ultimate column. Joint-life products need two ages. None of this lives in the table itself; it lives in the convention that produced the table.

A `MortalityTable` is a thin wrapper over the underlying `Table` that records those conventions explicitly. The wrapper does not re-implement table loading — it routes lookups through structure-aware logic so the `at(...)` call you write matches the convention the table was built under, and so a table built under one convention cannot be silently used as if it were another.

## Why Wrap a Table

The base `Table` class is a key-indexed lookup over a parquet or in-memory DataFrame. It will happily return a value for any keys you pass in, regardless of whether those keys make sense under the table's intended convention. Three failure modes that `MortalityTable` closes:

- **Age-basis confusion.** A table quoted on age last birthday will return rates that are silently off by half a year if you look it up using attained age nearest birthday. The base Table cannot tell. The wrapper validates the basis you supply against the basis the table was registered under.
- **Select-ultimate without clamping.** A 25-year select table with `select_period=24` rolls duration ≥ 24 into the ultimate column. Looking up `duration=30` against the raw Table either misses or returns whatever happens to be in row 30. The wrapper clamps duration at `select_period` automatically.
- **Joint-life with the wrong API.** A joint table needs two ages, in a known order. The wrapper rejects single-age lookups against joint tables and vice versa.

These checks fire at lookup time, against real data. They are the kind of checks an experienced reviewer applies to someone else's model — moving them into the type makes them automatic.

## Building a MortalityTable

```python
import polars as pl

from gaspatchio import ActuarialFrame, MortalityTable
from gaspatchio.assumptions import Table

# 1. Underlying Table — keys + value column. An aggregate base table
#    carries one rate per age (here, an illustrative 2017 CSO male-smoker
#    extract).
mort_source = pl.DataFrame(
    {
        "age": [30, 40, 50, 60, 70, 80],
        "mort_rate": [0.0011, 0.0021, 0.0049, 0.0118, 0.0301, 0.0802],
    }
)
mort_table = Table(
    name="cso_2017_male_smoker",
    source=mort_source,
    dimensions={"age": "age"},
    value="mort_rate",
)

# 2. Wrap it with the conventions the table was built under.
mortality = MortalityTable(
    table=mort_table,
    age_basis="age_last_birthday",
    structure="aggregate",
)
```

`age_basis` is one of the standard actuarial bases: `"age_last_birthday"`, `"age_nearest_birthday"`, `"age_next_birthday"`. `structure` is `"aggregate"`, `"select_ultimate"`, or `"joint"`. For select-ultimate, also pass `select_period=N` (typically 5, 15, or 25 depending on the table).

## The Three Structures

### Aggregate

One rate per age, regardless of how long the policy has been in force.

```python
agg = MortalityTable(
    table=mort_table,
    age_basis="age_last_birthday",
    structure="aggregate",
)

af = ActuarialFrame(pl.DataFrame({"policy_id": ["P001", "P002"], "age": [40, 60]}))
af.mort_rate = agg.at(age=af.age)
```

This is the simplest case and the most common one for valuation tables that come from regulator publications (e.g., 2017 CSO base tables in aggregate form).

### Select-Ultimate

Two rates per age — one for the "select" period after issue (where mortality is lower because the policyholder passed underwriting), one for the "ultimate" period after the select effect wears off. The wrapper clamps `duration` at `select_period` so durations beyond the select horizon roll into the ultimate row automatically.

```python
# A select-ultimate base table carries one rate per (age, duration) inside the
# select window; durations beyond select_period roll into the last row.
su_source = pl.DataFrame(
    {
        "age": [40] * 5 + [60] * 5,
        "duration": [0, 1, 2, 3, 4] * 2,
        "mort_rate": [
            0.0010, 0.0012, 0.0015, 0.0019, 0.0023,
            0.0100, 0.0112, 0.0125, 0.0139, 0.0154,
        ],
    }
)
su_table = Table(
    name="cso_2017_select",
    source=su_source,
    dimensions={"age": "age", "duration": "duration"},
    value="mort_rate",
)

sel = MortalityTable(
    table=su_table,
    age_basis="age_last_birthday",
    structure="select_ultimate",
    select_period=4,
)

af = ActuarialFrame(
    pl.DataFrame(
        {
            "policy_id": ["P001", "P002"],
            "age": [40, 60],
            # Per-period duration vectors. With select_period=4, durations 5 and 6
            # roll into the ultimate (last select) row automatically.
            "duration": [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]],
        }
    )
)
af.mort_rate = sel.at(age=af.age, duration=af.duration)
print(af.collect().select(["age", "mort_rate"]))
```

The clamp removes the `af.duration_capped = af.duration.clip(upper_bound=24)` boilerplate that select-ultimate lookups would otherwise require — the table's structure metadata makes the clamp automatic.

### Joint

Two ages, one rate. The wrapper takes `age_1` and `age_2` (in the order the table was built) and rejects accidental single-age lookups against a joint table.

```python
# A joint base table is keyed on two ages, in the order the table was built.
joint_source = pl.DataFrame(
    {
        "age_1": [60, 60, 65, 65],
        "age_2": [58, 62, 58, 62],
        "mort_rate": [0.018, 0.020, 0.022, 0.025],
    }
)
joint_table = Table(
    name="joint_last_survivor",
    source=joint_source,
    dimensions={"age_1": "age_1", "age_2": "age_2"},
    value="mort_rate",
)

jnt = MortalityTable(
    table=joint_table,
    age_basis="age_last_birthday",
    structure="joint",
)

af = ActuarialFrame(pl.DataFrame({"male_age": [60, 65], "female_age": [58, 62]}))
af.joint_mort = jnt.at(age_1=af.male_age, age_2=af.female_age)
```

## Worked Example: Aggregate and Select-Ultimate

```python
import polars as pl

from gaspatchio import ActuarialFrame, MortalityTable
from gaspatchio.assumptions import Table

# --- Aggregate lookup ---
agg_source = pl.DataFrame(
    {
        "age": [30, 40, 50, 60, 70, 80],
        "mort_rate": [0.001, 0.002, 0.005, 0.012, 0.030, 0.080],
    }
)
agg_table = Table(
    name="illustrative_aggregate",
    source=agg_source,
    dimensions={"age": "age"},
    value="mort_rate",
)
agg = MortalityTable(
    table=agg_table,
    age_basis="age_last_birthday",
    structure="aggregate",
)

af = ActuarialFrame(pl.DataFrame({"age": [30, 40, 50, 60]}))
af.mort_rate = agg.at(age=af.age)
print(af.collect())
```

```text
shape: (4, 2)
┌─────┬───────────┐
│ age ┆ mort_rate │
│ --- ┆ ---       │
│ i64 ┆ f64       │
╞═════╪═══════════╡
│ 30  ┆ 0.001     │
│ 40  ┆ 0.002     │
│ 50  ┆ 0.005     │
│ 60  ┆ 0.012     │
└─────┴───────────┘
```

```python
# --- Select-ultimate lookup with automatic duration clamping ---
sel_source = pl.DataFrame(
    {
        "age": [40] * 5 + [41] * 5 + [50] * 5,
        "duration": [0, 1, 2, 3, 4] * 3,
        "mort_rate": [
            0.0010, 0.0012, 0.0015, 0.0019, 0.0023,
            0.0011, 0.0013, 0.0016, 0.0020, 0.0024,
            0.0040, 0.0044, 0.0048, 0.0053, 0.0058,
        ],
    }
)
sel_table = Table(
    name="illustrative_select",
    source=sel_source,
    dimensions={"age": "age", "duration": "duration"},
    value="mort_rate",
)
sel = MortalityTable(
    table=sel_table,
    age_basis="age_last_birthday",
    structure="select_ultimate",
    select_period=4,
)

af2 = ActuarialFrame(
    pl.DataFrame(
        {
            "issue_age": [40, 41, 50],
            # Per-policy durations 0..6. The wrapper clamps any duration
            # > select_period (4) to the ultimate row automatically.
            "duration": [list(range(7)), list(range(7)), list(range(7))],
        }
    )
)
af2.mort_rate = sel.at(age=af2.issue_age, duration=af2.duration)
print(af2.collect())
```

```text
shape: (3, 3)
┌───────────┬─────────────┬────────────────────────────┐
│ issue_age ┆ duration    ┆ mort_rate                  │
│ ---       ┆ ---         ┆ ---                        │
│ i64       ┆ list[i64]   ┆ list[f64]                  │
╞═══════════╪═════════════╪════════════════════════════╡
│ 40        ┆ [0, 1, … 6] ┆ [0.001, 0.0012, … 0.0023]  │
│ 41        ┆ [0, 1, … 6] ┆ [0.0011, 0.0013, … 0.0024] │
│ 50        ┆ [0, 1, … 6] ┆ [0.004, 0.0044, … 0.0058]  │
└───────────┴─────────────┴────────────────────────────┘
```

The `duration=6` lookup against age 40 returns `0.0023` — the same rate as `duration=4`, because the wrapper clamped 5 and 6 to the select-period boundary. No `af.duration_capped = af.duration.clip(upper_bound=4)` step in your model code.

## When Conventions Conflict

If you supply an explicit `age_basis=` to `at(...)` that disagrees with the table's registered basis, the wrapper raises:

```python
af_basis = ActuarialFrame(pl.DataFrame({"age": [40, 60]}))
try:
    agg.at(age=af_basis.age, age_basis="age_nearest_birthday")
except ValueError as exc:
    print(exc)
    # ValueError: requested age_basis='age_nearest_birthday' but table's
    # age_basis is 'age_last_birthday'; cross-basis conversion is not
    # supported.
```

Cross-basis conversion (apply a half-year adjustment, look up under the converted basis) is not supported; supplying a different basis is treated as a configuration error rather than silently producing wrong rates.

## Governance: Mortality Basis Identity

Every MortalityTable produces a `source_sha()` that pins down the basis — the underlying table together with the conventions you've declared (age basis, structure, select period):

```python
agg.source_sha()
# 'sha256:e7ea7836978956b7ea610f97bcfae63ea039755c2693fa11639c31ce29854ca4'
```

Record this alongside each valuation. Swap to a different table, change the age basis, or extend the select period and the value changes — so a mortality basis change between quarters is visible the moment you compare runs, not on page seventeen of an experience study. Like the Schedule and Curve, the MortalityTable's identity rolls up into the rollforward's [overall version stamp](https://gaspatchio.dev/0.9.0/concepts/rollforward/inspection/index.md), so mortality basis changes show up in the model-level stamp without separate tracking.

## Runnable Companions

The patterns above are exercised end-to-end in the mini-VA tutorial:

- `bindings/python/gaspatchio/tutorials/level-3-mini-va-typed/steps/01-from-files/model.py` — aggregate table loaded from `mortality.parquet`
- `bindings/python/gaspatchio/tutorials/level-3-mini-va-typed/steps/02-select-mort/model.py` — select-ultimate with `select_period=24`, sex-based `table_id` lookup through the third dimension

The tutorial's README captures rough edges discovered while building those steps and is worth reading alongside this page.

## See Also

- [Assumption Tables](https://gaspatchio.dev/0.9.0/concepts/assumptions/index.md) — the underlying `Table` mechanics that MortalityTable wraps
- [Inspection](https://gaspatchio.dev/0.9.0/concepts/rollforward/inspection/index.md) — the MortalityTable's basis identity rolls up into the rollforward's overall version stamp

# Projections: Time-Dependent Calculations

Actuarial models are about projecting values forward through time. Every life product — term, whole life, universal life — steps through monthly or annual periods, computing cashflows, decrements, and accumulated values at each step. This page covers how Gaspatchio expresses those time-dependent calculations on list columns, where each list represents one policy's projection over time.

Two `.projection` accessors

This page covers the **column-level** accessor — `af.qx.projection.cumulative_survival()`, `af.growth.projection.accumulate()`, and so on — for list-column time-series operations. The **frame-level** `af.projection` accessor (`af.projection.set(...)`, `af.projection.rollforward(...)`, `af.projection.year_fractions()`) is covered in [Schedules](https://gaspatchio.dev/0.9.0/concepts/schedules/index.md) and [Rollforward](https://gaspatchio.dev/0.9.0/concepts/rollforward/index.md).

The snippets on this page all operate on the same small portfolio: two universal-life policies, each projected over a 24-month grid. Premiums, mortality rates, interest rates, and a starting account value are carried per policy, with the time axis held as a list in each cell.

```python
from gaspatchio import ActuarialFrame

N = 24  # 24 monthly projection periods

af = ActuarialFrame({
    "policy_id": ["P001", "P002"],
    "issue_age": [45, 52],
    "premium": [[1000.0] * N, [1500.0] * N],
    "interest_rate": [[0.04 / 12] * N, [0.04 / 12] * N],
    "qx": [[0.0008] * N, [0.0012] * N],           # monthly mortality rate
    "mort_rate_mth": [[0.0008] * N, [0.0012] * N],
    "death_benefit": [[100_000.0] * N, [150_000.0] * N],
    "av_init": [5_000.0, 8_000.0],                # starting account value (scalar)
    "av_pp_init": [5_000.0, 8_000.0],
    "maint_fee_rate": [0.012, 0.012],             # annual maintenance fee
    "inv_return_mth": [[0.003] * N, [0.003] * N], # monthly investment return
    "premiums": [[1000.0] * N, [1500.0] * N],
    "charges": [[50.0] * N, [75.0] * N],          # monthly fixed charges
})

print(af.collect().select(["policy_id", "issue_age", "av_init"]))
```

## Why Not Recursion?

Many actuarial modelling frameworks use recursive functions for time-dependent calculations. A typical recursive style uses timing strings such as `"BEF_PREM"` to mark sub-period checkpoints in an evaluation graph:

```python
# Recursive style — function calls itself
def av_pp_at(t, timing):
    if timing == "BEF_PREM":
        if t == 0:
            return av_pp_init()
        else:
            return av_pp_at(t-1, "BEF_INV") + inv_income_pp(t-1)
```

This is mathematically elegant. But recursive functions are confusing to debug — every change means stepping through layers of nested calls and mentally unwinding the recursion to see what's happening.

### The question that trips people up

> "Can you build a Universal Life model where Cost of Insurance depends on Account Value, which depends on Credited Interest, which depends on Account Value from last month?"

This *sounds* like a circular reference. Excel warns you about these. The FAST Standard says "never release a model with purposeful circularity."

But it's not circular at all. Account Value at time *t* depends on Account Value at time *t-1* — a chain through time, not a circle.

For *most* time-dependent calculations — survival probabilities, discount factors, simple account accumulation — the chain can be expressed as a closed-form cumulative product or sum. Consider survival probability:

**ₜpₓ = p₀ × p₁ × p₂ × ... × pₜ₋₁**

That's a **cumulative product**. No recursion needed. Gaspatchio lets you write time-dependent calculations this way.

There is a genuine escalation: when a charge at time *t* depends on the *current* account value (COI on net amount at risk, IUL floor/cap), the cumulative form breaks down. For these products, [Rollforward Methods](https://gaspatchio.dev/0.9.0/concepts/rollforward/index.md) handles within-period state-dependent calculations. The arc on this page leads up to that handover.

## The Insight

What looks like recursion in actuarial models is usually one of three things:

| Recursive form           | What it really is      | Gaspatchio equivalent          |
| ------------------------ | ---------------------- | ------------------------------ |
| `f(t) = f(t-1) × factor` | Cumulative product     | `cum_prod()`                   |
| `f(t) = f(t-1) + amount` | Cumulative sum         | `cum_sum()`                    |
| `f(t-1)`                 | Reference prior period | `projection.previous_period()` |

No loops. No circular references. Just express the math directly.

## Time-Shifting

These methods shift values forward or backward along the projection timeline. They are the building blocks for any calculation that references "last month's value" or "next month's value."

### `previous_period(fill_value=0.0)`

Returns the value from the previous time step — the row above in a spreadsheet.

```python
af.last_month_premium = af.premium.projection.previous_period(fill_value=0.0)
# [0.0, 1000, 1000, 1000, 1000]  from  [1000, 1000, 1000, 1000, 1000]
```

The first period has no predecessor, so `fill_value` controls what goes there (typically 0).

### `next_period(fill_value=0.0)`

Returns the value from the next time step. Useful for forward-looking calculations like surrender value at the next anniversary.

### `at_period(relative_period, fill_value=0.0)`

Arbitrary time offset — negative for past, positive for future. Generalises `previous_period` and `next_period`.

```python
# Two periods back
af.rate_two_months_ago = af.interest_rate.projection.at_period(-2)

# Three periods forward
af.rate_in_three_months = af.interest_rate.projection.at_period(3)
```

## Broadcasting to the Projection Axis

A per-policy attribute is a scalar — one occupation class, one product code, one smoker flag per policy. An assumption lookup keyed by that attribute *and* a time-varying key needs the attribute once per period. For numbers, arithmetic broadcasting handles this invisibly (`af.scalar * af.list`), but a string cannot ride an arithmetic operator — and re-encoding string-keyed dimensions as numeric codes just to broadcast them puts translation tables between the model and its assumptions.

### `broadcast_to_periods(like=None)`

Repeats a per-policy scalar of any dtype — strings, booleans, categoricals, numerics — into a list aligned with the projection axis.

```python
plans = ActuarialFrame({
    "policy_id": ["P001", "P002"],
    "occupation_class": ["WC1", "WC3"],   # per-policy string attribute
    "premium": [[100.0, 100.0, 100.0], [80.0, 80.0, 80.0]],
})

# One value per projection period, ready for a per-period lookup
plans.occ_pp = plans.occupation_class.projection.broadcast_to_periods(like=plans.premium)

print(plans.collect())
# occ_pp: [["WC1", "WC1", "WC1"], ["WC3", "WC3", "WC3"]]
```

On a frame with a projection timeline (`af.projection.set(...)`), the `month` axis is the default length source and `like=` can be omitted. On jagged timelines, or frames without one, name any list column to match with `like=` — each row's output takes that row's length.

This pairs with string-valued `when()` branches: `when(af.code_list == 0).then("PP23").otherwise("PP24")` produces a `List(String)` column directly, so per-period string codes flow into [conditionals](https://gaspatchio.dev/0.9.0/api/conditionals/index.md) and table lookups without numeric re-encoding.

## Cumulative Operations

### `cum_prod()` — Cumulative Product

Multiplicative accumulation — survival probabilities, compound growth, discount factors.

```python
from gaspatchio import ActuarialFrame

surv = ActuarialFrame({
    "policy_id": ["P001"],
    "annual_survival": [[0.99, 0.98, 0.97, 0.96, 0.95]],
})

# Cumulative survival probability: ₜpₓ = p₀ × p₁ × ... × pₜ
surv.cum_survival = surv.annual_survival.cum_prod()

print(surv.collect())
# cum_survival: [0.99, 0.9702, 0.9411, 0.9035, 0.8583]
```

### `cum_sum()` — Cumulative Sum

Additive accumulation — cumulative premiums, cumulative claims, account balances with deposits.

```python
paid = ActuarialFrame({
    "policy_id": ["P001"],
    "monthly_premium": [[100, 100, 100, 150, 150]],
})

paid.total_paid = paid.monthly_premium.cum_sum()
# total_paid: [100, 200, 300, 450, 600]
```

### `cumulative_survival()` — Mortality to Survival

The most common cumulative product in actuarial work: converting period mortality rates (qx) into cumulative survival probabilities using `tpx[t] = (1-qx[0]) × (1-qx[1]) × ... × (1-qx[t-1])`.

```python
af.survival = af.qx.projection.cumulative_survival()
# survival starts at 1.0, then compounds (1 - qx) each period:
# qx = [0.01, 0.02, 0.03]  →  survival = [1.0, 0.99, 0.9702]
```

The `rate_timing` parameter controls whether the rate at time *t* affects survival at time *t* (end of period) or time *t+1* (beginning of period). The `start_at` parameter sets the initial survival probability (default 1.0).

## Period Overrides

These methods modify values at specific time steps within a projection. Essential for modelling contractual changes, premium holidays, and benefit adjustments.

### `with_period(period, value)`

Override a single period's value. Supports negative indexing (e.g., `-1` for the last period).

```python
# Premium holiday at month 12
af.premium_adjusted = af.premium.projection.with_period(12, value=0)

# Maturity benefit at final period
af.benefit = af.death_benefit.projection.with_period(-1, value=10000)
```

### `with_periods(updates)`

Override multiple periods at once. Takes a dictionary of `{period: value}` pairs.

```python
# Premium holidays at months 6, 12, and 18
af.premium_adjusted = af.premium.projection.with_periods({6: 0, 12: 0, 18: 0})
```

## Worked Example: Universal Life Account Value

This is the "circular reference" problem traced end-to-end. The dependency:

```text
COI[t] depends on → Account Value[t]
Account Value[t] depends on → Credited Interest[t-1]
Credited Interest[t-1] depends on → Account Value[t-1]  ← PREVIOUS period
```

A chain through time, not a circle.

When the per-period growth and cashflows can be pre-computed as rates, the cumulative form expresses this directly:

```python
# Growth factor per period: after fees, after investment return
af.growth_factor = (1.0 - af.maint_fee_rate / 12.0) * (1.0 + af.inv_return_mth)

# Cumulative growth from t=0 to each t
af.cumulative_growth = af.growth_factor.cum_prod()

# Account value = initial × cumulative growth (shifted by one period)
af.av_pp_bef_prem = af.av_pp_init * af.cumulative_growth.projection.previous_period(fill_value=1.0)

# Survival probability — use the helper
af.tpx = af.mort_rate_mth.projection.cumulative_survival()
```

The whole accumulation reads in three lines instead of a recursive callback graph.

When charges depend on the current AV

The example above works when fees and returns can be pre-computed as rates. For products where charges depend on the current AV — like COI on net amount at risk — use [Rollforward Methods](https://gaspatchio.dev/0.9.0/concepts/rollforward/index.md) instead. The rollforward handles within-period state-dependent charges that can't be factored into a cumulative product.

## Escalation: `accumulate()` — the Linear Recurrence

The cumulative operations above cover pure multiplicative or additive accumulation. The standard AV accumulation mixes both:

```text
AV[t] = (AV[t-1] + Premium[t] − Charges[t]) × (1 + i[t])
```

That is a **linear recurrence** — `State[t] = State[t-1] × M[t] + A[t]`. `accumulate()` expresses this in a single call. You pre-compute the multiplicative and additive components in Python so the business logic stays readable; the per-policy time walk and the parallelism across policies are handled for you.

### Basic Example

```python
from gaspatchio import ActuarialFrame

# Two policies: different starting AV, same growth and cashflows
data = {
    "av_init": [1000.0, 2000.0],
    "growth": [[1.01, 1.01, 1.01], [1.02, 1.02, 1.02]],
    "net_flow": [[50.0, 50.0, 50.0], [100.0, 100.0, 100.0]],
}
demo = ActuarialFrame(data)

demo.av = demo.growth.projection.accumulate(
    initial="av_init",
    multiply="growth",
    add="net_flow",
)

print(demo.collect()["av"].to_list())
# [[1060.0, 1120.6, 1181.806], [2140.0, 2282.8, 2428.456]]
```

Each time step: `AV[t] = AV[t-1] × growth[t] + net_flow[t]`.

### Rearranging the AV formula

The standard AV accumulation `AV[t] = (AV[t-1] + Premium[t] − Charges[t]) × (1 + i[t])` fits `State[t] = State[t-1] × M[t] + A[t]` after a small algebra step:

- **M[t]** = `(1 + i[t])` — the growth factor
- **A[t]** = `(Premium[t] − Charges[t]) × (1 + i[t])` — net cashflow, grown by interest

```python
growth = 1 + af.interest_rate
net_flow_grown = (af.premiums - af.charges) * growth

af.av = af.interest_rate.projection.accumulate(
    initial=af.av_init,
    multiply=growth,
    add=net_flow_grown,
)
```

The business logic is what you see — three column expressions and one `accumulate(...)` call.

### Performance

A single `accumulate()` call carries the per-policy time axis, and the portfolio runs in parallel across all policies. Headline runtimes on a typical workstation:

| Scale                         | Wall time            |
| ----------------------------- | -------------------- |
| 1,000 policies × 240 months   | under 1 millisecond  |
| 10,000 policies × 360 months  | a few milliseconds   |
| 100,000 policies × 360 months | tens of milliseconds |

That covers the AV accumulation step itself; full models layer further calculations on top. The point is that the time-step loop is no longer the bottleneck — building inputs, running scenarios, and writing outputs dominate the wall time.

### Parameters

| Parameter  | Type                                    | Description                                                                          |
| ---------- | --------------------------------------- | ------------------------------------------------------------------------------------ |
| `initial`  | str, Expr, ExpressionProxy, ColumnProxy | Initial state per policy (scalar column). Broadcasts when length is 1.               |
| `multiply` | str, Expr, ExpressionProxy, ColumnProxy | Multiplicative factor per time step (list column).                                   |
| `add`      | str, Expr, ExpressionProxy, ColumnProxy | Additive flow per time step (list column). Inner list lengths must match `multiply`. |

All parameters accept column names as strings, Polars expressions, or gaspatchio proxy objects.

## When `accumulate()` Isn't Enough

`accumulate()` handles `State[t] = State[t-1] × M + A` — the case where every input can be pre-computed before the time walk starts. For products with **state-dependent charges** — where the charge at time *t* depends on the accumulated value at *t* — see [Rollforward Methods](https://gaspatchio.dev/0.9.0/concepts/rollforward/index.md). Common examples: COI on net amount at risk, tiered management charges, IUL crediting with floor and cap, and multi-state products like VA + GMDB.

The rollforward declares within-period steps as a chain that reads like the product spec. It sits on an explicit projection grid (`af.projection.set(...)`), and the chain is compiled before the per-period values are read back onto the frame:

```python
from datetime import date
from gaspatchio import (
    ActuarialFrame,
    RollforwardCollector,
    Schedule,
    compile_rollforward,
)

ul = ActuarialFrame({
    "policy_id": ["P001", "P002"],
    "av_init": [5_000.0, 8_000.0],
    "premium": [[1000.0] * N, [1500.0] * N],
    "coi_rate": [[0.001] * N, [0.0015] * N],
    "sum_assured": [[100_000.0] * N, [150_000.0] * N],
    "admin_rate": [[0.012 / 12] * N, [0.012 / 12] * N],
    "interest_rate": [[0.04 / 12] * N, [0.04 / 12] * N],
})
sched = Schedule.from_calendar_grid(
    start_date=date(2025, 1, 31), n_periods=N, frequency="1M"
)
ul = ul.projection.set(schedule=sched)

b = ul.projection.rollforward(states={"av": ul["av_init"]})
(
    b["av"]
    .add(ul["premium"], label="Premium")
    .deduct_nar(ul["coi_rate"], death_benefit=ul["sum_assured"], label="COI")
    .charge(ul["admin_rate"], label="Admin")
    .grow(ul["interest_rate"], label="Interest")
    .floor(value=0.0)
)

compiled = compile_rollforward(b)
ul.av = RollforwardCollector(compiled).expr_for("av")
print(ul.collect().select(["policy_id", "av"]))
```

## The Other Direction: `prospective_value()`

Calculates the present value of future cashflows at each projection period using backward recursion. Essential for reserve calculations, embedded value, and profit testing.

```python
# PV of death benefits at 5% discount rate
af.pv_death = af.death_benefit.projection.prospective_value(discount_rate=0.05)
```

Specify either `discount_rate` (constant or per-period) or `discount_factor` (pre-computed v^t values), but not both. The `timing` parameter controls whether cashflows fall at the end or beginning of each period.

## Why This Works (For the Curious)

The recursive form and cumulative form are algebraically identical. The proof for a simple accumulation:

**Recursive definition:**

```text
AV[0] = AV_init
AV[t] = AV[t-1] × factor[t-1]
```

**Expanding the recursion:**

```text
AV[1] = AV[0] × factor[0]
AV[2] = AV[1] × factor[1] = AV[0] × factor[0] × factor[1]
...
AV[t] = AV[0] × factor[0] × factor[1] × ... × factor[t-1]
```

**Closed form:**

```text
AV[t] = AV_init × ∏(factor[i] for i in 0..t-1)
```

That product is exactly what `cum_prod()` computes. The `previous_period()` shift handles the "up to t-1" part.

The same pattern applies to survival probabilities:

```text
ₜpₓ = ₜ₋₁pₓ × pₓ₊ₜ₋₁           (recursive)
ₜpₓ = ∏(pₓ₊ᵢ for i in 0..t-1)  (cumulative product)
```

Actuaries have used the closed form in theory for centuries. Gaspatchio just lets you write it that way in code.

# Running Models Across Scenarios

A scenario run takes one model and exercises it under many sets of assumptions — base, mortality up, lapse down, interest stress, a thousand stochastic draws. The output is a small set of summary numbers (SCR, BEL, worst case) plus, ideally, an audit chain a regulator can verify.

Two layers do this work:

- **A typed plan** (`ScenarioRun`) — shocks, base tables, aggregations bundled together with a `source_sha` and an opt-in audit sidecar. The path you use when the run will be reproduced later.
- **A low-level helper** (`with_scenarios`) — cross-joins your model points with a scenario list. The path you use for one-shot exploration when you don't need governance.

Most production scenario work runs through the typed plan. The low-level helper is the escape hatch.

______________________________________________________________________

## The primary path: `ScenarioRun`

A `ScenarioRun` carries your shocks, base tables, and aggregations as a single value. You build it, run it, save it to YAML, hand it to model risk — they reload, rerun, and reproduce your numbers byte-for-byte.

Base tables are **scenario-invariant** — the run adds the scenario axis itself, stacking each table across your scenarios with the shocks applied. The scenario story lives in the shocks, where model risk can read it, not hidden in the table's rows:

```python
import polars as pl
from gaspatchio.assumptions import Table
from gaspatchio.frame import ActuarialFrame
from gaspatchio.scenarios import ScenarioRun, Sum
from gaspatchio.scenarios.shocks import AdditiveShock


disc_rates = Table(
    name="disc_rates",
    source=pl.DataFrame({
        "year": [0, 1],
        "rate": [0.03, 0.03],
    }),
    dimensions={"year": "year"},
    value="rate",
)


def policies():
    return ActuarialFrame({
        "policy_id": [1, 2, 3],
        "premium":   [100.0, 200.0, 300.0],
        "year":      [0, 1, 0],
    })


def model(af, *, tables, drivers=None):
    rate = tables["disc_rates"].lookup(
        scenario_id=af["scenario_id"],
        year=af["year"],
    )
    return af.with_columns((af["premium"] / (1.0 + rate)).alias("pv"))


plan = ScenarioRun(
    shocks={
        "BASE": [],
        "UP":   [AdditiveShock(delta=0.02)],    # 3% -> 5%
        "DOWN": [AdditiveShock(delta=-0.02)],   # 3% -> 1%
    },
    base_tables={"disc_rates": disc_rates},
    aggregations=(Sum("pv").alias("total_pv"),),
)

result = plan.run(policies(), model, batch_size=1)
print(plan.source_sha())                  # sha256:8a7466213697...
print(result.aggregations["total_pv"])    # 1748.01
```

The three pieces that make the typed-plan path useful:

- **[Aggregators](https://gaspatchio.dev/0.9.0/concepts/scenarios/aggregators/index.md)** — 14 built-in reducers (Sum, Mean, CTE, ArgMax, …) with `.alias()`, `.over()`, `.of()` modifiers. The aggregator carries its own column and reduction; you don't hand-roll a `group_by`.
- **[Scenario Run](https://gaspatchio.dev/0.9.0/concepts/scenarios/scenariorun/index.md)** — typed plan + audit sidecar + YAML round-trip + master-seed determinism.
- **[Custom Aggregators](https://gaspatchio.dev/0.9.0/concepts/scenarios/custom-aggregators/index.md)** — write your own (Skewness, TVaR, anything mergeable) and have it round-trip through governance the same way the built-ins do.

______________________________________________________________________

## The low-level helper: `with_scenarios`

`with_scenarios(af, scenario_ids)` cross-joins your model points with a scenario list and returns an `ActuarialFrame` with a `scenario_id` column. You write the model the way you'd write a single-scenario model and read `af["scenario_id"]` from your assumption lookups; aggregation is on you with raw polars.

```python
import gaspatchio as gs
from gaspatchio import ActuarialFrame

af = ActuarialFrame(pl.read_parquet("model_points.parquet"))
af = gs.with_scenarios(af, ["BASE", "UP", "DOWN"])

# af now has policies × 3 rows with a scenario_id column added.
# Run your model normally, then group_by("scenario_id") at the end.
```

This pattern is fine for exploratory work — interactive sessions, one-off stress checks, ad-hoc reporting where you'd rather inline the aggregation than declare it. It does *not* carry a `source_sha`, doesn't write an audit sidecar, and won't survive a YAML round-trip — so reach for it when the run is disposable.

When the analysis settles, promote it to a `ScenarioRun`. The model function reshapes slightly (it gains `*, tables, drivers=None` kwargs) but the projection logic stays the same.

______________________________________________________________________

## Loading scenario-varying assumptions

Most assumptions stay the same across scenarios (mortality, lapse, premium rates). The economic ones — discount rates, equity returns, inflation — typically vary. Three ways to load them, all returning a `Table` keyed by `scenario_id`.

One boundary to respect: `ScenarioRun` — and any shocks-dict run — stacks every base table across scenarios itself, so its `base_tables` must be scenario-invariant, with the variation expressed as shocks. The scenario-keyed tables below belong to the other two shapes: pass them to `for_each_scenario` with a plain scenario-id list (or a drivers dict), and `base_tables` reaches your model untouched, ready for a `scenario_id=` lookup.

### Single table with `scenario_id` dimension

If your data is in one file with a `scenario_id` column:

```python
disc_rates = Table(
    name="disc_rates",
    source="disc_rates.parquet",
    dimensions={
        "scenario_id": "scenario_id",
        "year": "year",
    },
    value="disc_rate_ann",
)

# Inside model_fn:
rate = disc_rates.lookup(scenario_id=af["scenario_id"], year=af["year"])
```

### Separate files per scenario

If your scenarios live in separate files (typical for ESG output):

```python
disc_rates = Table.from_scenario_files(
    scenario_files={
        "BASE": "scenarios/BASE/disc_rates.parquet",
        "UP":   "scenarios/UP/disc_rates.parquet",
        "DOWN": "scenarios/DOWN/disc_rates.parquet",
    },
    scenario_column="scenario_id",
    dimensions={"year": "year"},
    value="disc_rate_ann",
    name="disc_rates",
)
```

Each file is tagged with its scenario_id, concatenated, and exposed as one `Table` keyed by `(scenario_id, year)`.

### Template-based loading

When file naming follows a pattern:

```python
disc_rates = Table.from_scenario_template(
    path_template="scenarios/{scenario_id}/disc_rates.parquet",
    scenario_ids=["BASE", "UP", "DOWN"],
    scenario_column="scenario_id",
    dimensions={"year": "year"},
    value="disc_rate_ann",
)
```

Equivalent to `from_scenario_files`, more compact.

All three return a `Table` you pass as `base_tables` to `for_each_scenario` alongside the matching scenario-id list — the loop hands it to your model untouched. Keep these out of `ScenarioRun`, whose stacker would collide with the scenario axis the table already carries.

______________________________________________________________________

## What to read next

| Doing                                         | Read                                                                                              |
| --------------------------------------------- | ------------------------------------------------------------------------------------------------- |
| Choosing an aggregator                        | [Aggregators](https://gaspatchio.dev/0.9.0/concepts/scenarios/aggregators/index.md)               |
| Building a reproducible run                   | [Scenario Run](https://gaspatchio.dev/0.9.0/concepts/scenarios/scenariorun/index.md)              |
| Adding a metric that's not built in           | [Custom Aggregators](https://gaspatchio.dev/0.9.0/concepts/scenarios/custom-aggregators/index.md) |
| Stress shocks (multiply, add, clip, pipeline) | [Shock Operations](https://gaspatchio.dev/0.9.0/concepts/scenarios/shocks/index.md)               |
| Memory at scale                               | [Performance](https://gaspatchio.dev/0.9.0/concepts/scenarios/performance/index.md)               |
| Asking what-if from natural language          | [What-If Analysis](https://gaspatchio.dev/0.9.0/concepts/scenarios/what-if/index.md)              |

# Schedules

## The Time Axis Is Part of the Model

Every projection sits on a sequence of period boundaries — month-ends or year-ends from inception, anniversary dates that anchor commission and ratchet logic, calendar-aware adjustments for business-day conventions. Getting these wrong is how you accidentally pay a quarterly premium thirteen times in a 12-month projection or miss an anniversary that should have triggered a guaranteed step-up.

Gaspatchio attaches the time axis to the frame. You declare it once with `af.projection.set(...)`, and every period-boundary, year-fraction, and anniversary derives consistently from the same source.

## Setting the Projection: `af.projection.set(...)`

The primary entry point is `af.projection.set(...)`. It takes a valuation date, an end condition, and a frequency, and returns a new frame with the projection metadata attached:

```python
import datetime as dt
from gaspatchio import ActuarialFrame

af = ActuarialFrame({
    "policy_id":  ["P001", "P002", "P003"],
    "issue_age":  [30, 45, 60],
    "policy_inception": [dt.date(2020, 6, 15), dt.date(2018, 3, 1), dt.date(2023, 1, 1)],
})

af = af.projection.set(
    valuation_date=dt.date(2025, 1, 1),
    until="maximum_age",
    until_value=100,
    frequency="monthly",
)
```

`set(...)` produces a new frame carrying the projection — assign it back to `af` so the projection comes with you. Three columns appear on the frame after this call: `projection_start_date`, `projection_end_date`, `num_proj_months`.

Other end conditions: `until="term_months"` for a fixed number of periods, `until="next_anniversary"` for products whose horizon is a contract anniversary rather than a calendar duration, `until="end_date"` for an explicit calendar date. The `until_value` accepts an `int`, a column name (string), or a `pl.Expr` for per-policy values.

Frequency strings accept either English (`"monthly"`, `"quarterly"`, `"annual"`) or Schedule shorthand (`"1M"`, `"3M"`, `"1Y"`).

## Opt-in Methods on `af.projection`

Per-period quantities — year fractions, cumulative time, in-force flags, anniversary indicators — are available on `af.projection` when you ask for them. Add them to the frame only if you want them to appear in the final output:

```python
af.year_fractions = af.projection.year_fractions()
af.t_years        = af.projection.t_years()
af.in_force       = af.projection.is_in_force()
```

| Method                              | Returns                                                               | Use For                                                    |
| ----------------------------------- | --------------------------------------------------------------------- | ---------------------------------------------------------- |
| `af.projection.year_fractions()`    | per-period width using the bound day count (length `n_periods`)       | discount factor inputs, period-weighted aggregations       |
| `af.projection.t_years()`           | cumulative time-since-start at each boundary (length `n_periods + 1`) | natural input to `Curve.discount_factor(t)`                |
| `af.projection.is_in_force()`       | `True` where the period is **active** (length `n_periods`)            | actuarial expressions reading "is this policy alive at t?" |
| `af.projection.contract_boundary()` | `True` where the period is **terminated** (length `n_periods`)        | rollforward boundary indicator only                        |
| `af.projection.period_dates()`      | per-period boundary dates                                             | calendar-aware reporting, anniversary alignment            |

Most period-by-period roll-ups condition on whether the policy is in force at duration `t` — exposure aggregations, expected-lives counts, premium and claim runoffs. `af.projection.is_in_force()` produces that mask directly: `True` where the policy is in force, length `n_periods`. Reach for it whenever the calculation needs to ask "is this policy still here at `t`?".

`af.projection.contract_boundary()` returns the same information with the truth flipped — `True` where the policy has terminated. When you pass a boundary mask to `af.projection.rollforward(contract_boundary=...)`, the projection treats the first `True` period as termination: values from that period onward zero out. That's the shape `contract_boundary()` gives you. Everywhere else, prefer `is_in_force()`.

## The Typed Path: `Schedule`

A `Schedule` is your projection grid — valuation date, frequency, term, calendar, day-count — as a named, reusable object. `af.projection.set(valuation_date=..., until=..., frequency=...)` builds one for you in the common case and you never see it. When you want the grid to be explicit — shared across the term and annuity books, pinned to a specific quarter-end so its identity doesn't drift between runs, recorded in the audit file — build it directly and pass it in.

```python
from gaspatchio import ActuarialFrame

# Term book: three 20-year policies with per-policy remaining terms.
af_term = ActuarialFrame({
    "policy_id":   ["T001",  "T002",  "T003"],
    "issue_age":   [32,      40,      55],
    "sum_assured": [500_000, 300_000, 250_000],
    "remaining_term_months": [180, 120, 60],
})

# Annuity book: two payout annuities in the same quarterly run.
af_ann = ActuarialFrame({
    "policy_id":     ["A001",   "A002"],
    "issue_age":     [62,       68],
    "account_value": [250_000,  180_000],
})
```

```python
from datetime import date
from gaspatchio import Schedule

sched = Schedule.from_calendar_grid(
    start_date=date(2025, 1, 31),
    n_periods=240,
    frequency="1M",
)

af_term = af_term.projection.set(schedule=sched)
af_ann  = af_ann.projection.set(schedule=sched)   # share grid across books
```

When to reach for the typed path:

- Sharing the same grid across multiple frames (term + annuity book reconciliation, riders + base contract, sensitivity scenarios)
- Locking the grid down between valuations so you can prove next quarter's run used the same time axis as this quarter's — same period boundaries, same calendar conventions — and so any unintended drift shows up immediately rather than in your movement analysis
- Constructing per-policy schedules with `Schedule.from_inception(inception_column=..., n_periods=..., frequency=...)`

Whichever way you built the grid, the audit record is the same. `af.projection.canonical_form()` and `af.projection.source_sha()` come out identically whether `set(...)` built the grid from kwargs or you passed in an explicit `Schedule`. So you can switch between the two without your quarter-over-quarter version stamps shifting.

```python
af.projection.canonical_form()
# {'kind': 'from_calendar_grid', 'n_periods': 240, 'frequency': '1M',
#  'calendar': 'NullCalendar', 'convention': 'Unadjusted',
#  'day_count': 'OneTwelfth', 'anchor': 'month_end', 'start_date': '2025-01-31'}

af.projection.source_sha()
# 'sha256:7a12c74d650ea13f698fc6f7fe15a7da97eb52d7e268d8985d9c73b3ce981a52'
```

Record `source_sha()` alongside each quarterly run. If it changes between quarters without a deliberate release, the time axis has drifted — and your movement analysis has a contributor that nobody put there on purpose.

## Calendars and Day Counts

The bound Schedule binds three calendar-discipline choices, each with a sensible default. Pass them as kwargs to `Schedule.from_calendar_grid(...)` if you need to override:

| Choice       | Default                                             | When to override                                                                                                           |
| ------------ | --------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- |
| `calendar`   | `NullCalendar` (every day is a business day)        | Cross-border products, bond-coupon-style settlement, any model whose anniversaries must skip holidays                      |
| `convention` | `Unadjusted` (no business-day adjustment)           | Insurance contracts that defer benefit payments to the next business day; reinsurance with currency-specific conventions   |
| `day_count`  | `OneTwelfth` (each month is exactly 1/12 of a year) | Insurance products quoting under Actual/365 or Actual/Actual; reinsurance using Actual/360; bond-style models using 30/360 |

Available calendars: `NullCalendar`, `TARGET` (eurozone settlement), `UnitedKingdom`, `UnitedStates`, plus `JointCalendar(c1, c2)` for cross-border products and `BespokeCalendar(holidays=...)` for bespoke holiday lists.

Available day counts: `OneTwelfth`, `Actual360`, `Actual365Fixed`, `ActualActualISDA`, `Thirty360`. These match the conventions you would see in a product SOA filing or a reinsurance treaty.

```python
from datetime import date

from gaspatchio import Schedule
from gaspatchio.schedule import (
    UnitedStates, BusinessDayConvention, ActualActualISDA,
)

sched = Schedule.from_calendar_grid(
    start_date=date(2025, 1, 31),
    n_periods=120,
    frequency="1M",
    calendar=UnitedStates(),
    convention=BusinessDayConvention.MODIFIED_FOLLOWING,
    day_count=ActualActualISDA(),
)
af = af.projection.set(schedule=sched)
```

For the typical life-insurance model — monthly grid, no calendar adjustment, OneTwelfth day count — the defaults are correct and you only need `valuation_date`, `until`, `until_value`, and `frequency` on `af.projection.set(...)`.

## Worked Example: Anniversary Recognition

A monthly model where commission is paid on every policy anniversary. The Schedule produces an anniversary indicator — `True` at each anniversary period, `False` elsewhere — and the chain pays a fixed commission whenever the indicator fires.

```python
from datetime import date

from gaspatchio import Schedule

# Three-year monthly schedule anchored to month-end Jan 31.
sched = Schedule.from_calendar_grid(
    start_date=date(2025, 1, 31),
    n_periods=36,
    frequency="1M",
)

# Period boundaries — start, then end of each period.
dates = sched.period_dates()
print("Inception:    ", dates[0])
print("Final eop:    ", dates[-1])
print("Total bounds: ", len(dates))   # n_periods + 1

# Anniversary recognition — True at the end of each policy year.
mask = sched.anniversary_mask()
anniv_indices = [i for i, m in enumerate(mask) if m]
print("Anniversaries:", [dates[i] for i in anniv_indices])
```

```text
Inception:     2025-01-31
Final eop:     2028-01-31
Total bounds:  37
Anniversaries: [datetime.date(2025, 12, 31), datetime.date(2026, 12, 31), datetime.date(2027, 12, 31)]
```

The indicator is `True` at indices 11, 23, 35 — one anniversary per twelve-month period, on the contract anniversary in December. Feed it as a list column to a `.ratchet(when=...)` argument and the rollforward only ratchets on those periods. Or feed it through Polars conditionals to gate any per-period cash flow.

## Per-Policy Grids: `Schedule.from_inception`

Where every policy has its own inception date and anniversary recognition needs to be per-row, build a Schedule that references an input column. When the rollforward is prepared, it produces a per-policy boundary grid from that column.

```python
sched = Schedule.from_inception(
    inception_column="policy_inception",
    n_periods=240,
    frequency="1M",
)
af = af.projection.set(schedule=sched)
```

The `inception_column` must be a `Date` column on the input frame. Anniversary indicators for this Schedule are intrinsic — the inception date itself is the anchor, so there is no `anchor=` parameter.

For per-policy projection horizons without the typed path, pass a column name to `until_value`:

```python
import datetime as dt
from gaspatchio import ActuarialFrame

# Rebuild af with per-policy remaining terms for the until_value example below.
af = ActuarialFrame({
    "policy_id":  ["P001",                  "P002",                "P003"],
    "issue_age":  [30,                      45,                    60],
    "policy_inception": [dt.date(2020, 6, 15), dt.date(2018, 3, 1), dt.date(2023, 1, 1)],
    "remaining_term_months": [180,          120,                   60],
})
```

```python
af = af.projection.set(
    valuation_date=dt.date(2025, 1, 1),
    until="term_months",
    until_value="remaining_term_months",   # column name — per-policy
    frequency="monthly",
)
```

## Schedule Identity in the Audit File

The Schedule's identity rolls up into the rollforward's [overall version stamp](https://gaspatchio.dev/0.9.0/concepts/rollforward/inspection/index.md), so a change to the time axis shows up as a model-level change without you having to track it separately.

```python
sched.canonical_form()
sched.source_sha()
# Same values as af.projection.canonical_form() / af.projection.source_sha()
# when af.projection.set(schedule=sched) was used.
```

## Runnable Companions

The patterns above are exercised end-to-end in the typed mini-VA tutorial:

- `bindings/python/gaspatchio/tutorials/level-3-mini-va-typed/base/model.py` — `from_calendar_grid` with `OneTwelfth` day count, feeding `Curve.discount_factor` via cumulative year fractions
- `bindings/python/gaspatchio/tutorials/level-3-mini-va-typed/steps/07-anniversary-aware/model.py` — `until="next_anniversary"` for per-policy anniversary commissions

The tutorial's README captures rough edges discovered while building those steps and is worth reading alongside this page.

## See Also

- [Curves](https://gaspatchio.dev/0.9.0/concepts/curves/index.md) — `Curve.discount_factor(t)` consumes `af.projection.t_years()`
- [Multi-State Rollforwards](https://gaspatchio.dev/0.9.0/concepts/rollforward/multi-state/index.md) — anniversary indicator drives ratchet timing
- [Inspection](https://gaspatchio.dev/0.9.0/concepts/rollforward/inspection/index.md) — the Schedule's identity rolls up into the rollforward's overall version stamp

# Integrating Custom Python Code

As an actuary using Gaspatchio, you might have existing Python functions or complex logic you want to integrate into your models. Perhaps you have a specific benefit calculation, a complex decrement logic, or a custom reserving method implemented in Python.

Gaspatchio provides two primary ways to incorporate this custom logic into the `ActuarialFrame` workflow:

1. **Direct Application (`.apply`)**: For quick, one-off use cases or simple functions.
1. **Accessor Plugins**: For more complex, reusable logic that benefits from better organization and integration.

## 1. Direct Application with `.apply()`

If you have a relatively simple Python function that operates on a single column's data element-wise, the quickest way to use it is via the `.apply()` method on a column proxy.

Let's say you have a Python function to calculate a simple bonus amount based on the policy duration:

```python
# Your existing Python function
def calculate_bonus(duration: int) -> float:
    if duration <= 5:
        return 0.0
    elif duration <= 10:
        return 50.0
    else:
        return 100.0 + (duration - 10) * 5.0
```

You can apply this directly within your model definition:

```python
# docs-skip
import polars as pl
from gaspatchio import ActuarialFrame

# Assume 'af' is your ActuarialFrame with a 'policy_duration' column
# af = ActuarialFrame(...)

# Apply the custom Python function
# Note: We provide a return_dtype for better performance and type stability
af.bonus_amount = af.policy_duration.apply(
    calculate_bonus, return_dtype=pl.Float64
)

result = af.collect()
print(result)
```

**Pros:**

- **Simple:** Very straightforward for existing functions.
- **Quick:** No extra setup required for one-off calculations.

**Cons:**

- **Performance:** Python function execution can be slower than native Polars/Gaspatchio operations, especially for large datasets. Providing `return_dtype` helps, but it won't be as fast as a pure expression. For performance-critical code, rewrite using native Polars expressions or within an accessor plugin.
- **Readability:** Can clutter model logic if many complex `.apply` calls are used.
- **Reusability:** Less discoverable and reusable across different models compared to plugins.
- **Limited Scope:** Primarily designed for element-wise operations on single columns.

Use `.apply()` when you need a quick integration and the performance impact is acceptable, or when prototyping logic before potentially converting it into a more optimized expression or plugin.

Performance Considerations with `.apply()`

Using `.apply()` executes your Python function row by row. This involves overhead for each element (calling the Python interpreter, type checking, etc.) and prevents vectorized optimizations that operate on entire columns simultaneously. As a result, it can be **orders of magnitude slower** than equivalent logic written using native Polars/Gaspatchio expressions, especially on large datasets.

You might see a `PerformanceWarning` when using `.apply()` similar to this:

```text
PerformanceWarning: Applying a Python function 'your_function_name' using map_elements. This is potentially slow.
For better performance, consider using Polars expressions directly.
```

While convenient for quick tests or simple logic, relying heavily on `.apply()` for core calculations will significantly impact your model's performance. It's strongly recommended to rewrite the logic using native Polars expressions or within an accessor plugin for production use, as shown in the next section.

## 2. Accessor Plugins (Recommended for Reusability)

If your custom logic is more complex, will be reused across different models, or involves multiple related calculations, creating an **accessor plugin** is the recommended approach.

Accessor plugins extend `ActuarialFrame` (or its column/expression proxies) with custom namespaces. Think of the built-in `.dt` (for dates) or `.str` (for strings) namespaces in Polars – plugins let you create your own, like `.mortality` or `.reserving`.

### Why Create a Plugin?

- **Organization:** Group related custom calculations under a single namespace (e.g., `af.premium.finance.present_value(...)`).
- **Reusability:** Define logic once and use it across multiple models or share it with colleagues.
- **Readability:** Keeps model definitions cleaner by encapsulating complex logic within accessor methods.
- **Discoverability:** Makes custom functions easily discoverable via standard attribute access (and `dir()`).
- **Potential for Optimization:** Accessor methods can be written to leverage efficient Polars expressions internally.

### Creating a Simple Column Accessor

Let's adapt our `calculate_bonus` function into a reusable column accessor plugin.

**Step 1: Define the Accessor Class**

Create a Python file (e.g., `my_company_accessors.py`) and define your class:

```python
# docs-skip
# my_company_accessors.py
import polars as pl
from gaspatchio import ActuarialFrame, ColumnProxy, ExpressionProxy
from gaspatchio.frame.registry import register_accessor

class BaseAccessor:
    """Optional base class for convenience."""
    def __init__(self, obj):
        # obj will be the ColumnProxy or ExpressionProxy instance
        self._obj = obj

@register_accessor("bonus", kind="column") # Register as .bonus for columns/expressions
class BonusAccessor(BaseAccessor):

    def amount(self) -> ExpressionProxy:
        """Calculates the bonus amount based on the proxied duration column."""
        # We use Polars expressions *inside* the accessor for performance
        duration_expr = self._obj # self._obj is the duration column/expression proxy

        bonus_expr = (
            pl.when(duration_expr <= 5).then(0.0)
            .when(duration_expr <= 10).then(50.0)
            .otherwise(100.0 + (duration_expr - 10) * 5.0)
            .cast(pl.Float64) # Ensure consistent output type
        )
        # Important: Return an ExpressionProxy
        # We assume self._obj has a ._parent attribute (true for Column/ExpressionProxy)
        return ExpressionProxy(bonus_expr, self._obj._parent)

    def is_eligible(self, threshold: int = 5) -> ExpressionProxy:
         """Checks if bonus is eligible based on duration."""
         duration_expr = self._obj
         eligibility_expr = duration_expr > threshold
         return ExpressionProxy(eligibility_expr, self._obj._parent)


# IMPORTANT: Ensure this module (my_company_accessors.py) is imported somewhere
# in your application *after* gaspatchio is imported.
# e.g., in __init__.py or main.py:
# import my_company_accessors
```

**Key Points:**

- `@register_accessor("bonus", kind="column")`: This decorator registers the `BonusAccessor` class. It will be available as `.bonus` on `ColumnProxy` and `ExpressionProxy` instances.
- `__init__(self, obj)`: Stores the proxy object (`ColumnProxy` or `ExpressionProxy`) the accessor is attached to.
- `amount(self)`: Implements the bonus logic using efficient Polars `when/then/otherwise` expressions instead of a Python function. It returns a new `ExpressionProxy`.
- Returning `ExpressionProxy`: Accessor methods that perform calculations should generally return `ExpressionProxy` objects to keep the operations within the Gaspatchio/Polars expression system, so they fold into the same optimized plan as the rest of the model.

**Step 2: Import Your Accessor Module**

Somewhere in your project (e.g., your main script or a relevant `__init__.py`), make sure to import the module containing your accessor definition. This triggers the registration decorator.

```python
# docs-skip
# main_model.py
import polars as pl
from gaspatchio import ActuarialFrame
import my_company_accessors # <--- Import to register .bonus accessor

af = ActuarialFrame({ "policy_duration": [3, 7, 12] })

# Use the accessor!
af.bonus_amount = af.policy_duration.bonus.amount()
af.is_bonus_eligible = af.policy_duration.bonus.is_eligible()
# You can chain accessors with other operations
af.eligible_bonus = af.is_bonus_eligible * af.bonus_amount

print(af.collect())
```

### Frame Accessors and Entry Points

You can also create `frame` accessors (`kind="frame"`) that attach to the `ActuarialFrame` itself, useful for portfolio-level calculations.

Furthermore, if you are developing a package of reusable actuarial components, you can use **entry points** to make your accessors automatically discoverable when someone installs your package, without requiring them to explicitly import your accessor module.

These are more advanced topics covered in the technical reference documentation. For most users integrating their own project-specific code, the `@register_accessor` decorator provides the best balance of organization and ease of use.

Choose the method that best suits the complexity and reusability needs of your custom Python code. For simple, infrequent use, `.apply()` is sufficient. For structured, reusable, and potentially performance-critical logic, invest the time to create an accessor plugin.

# Extending the Rust Core

Gaspatchio's performance comes from its Rust core, which uses [Polars expressions](https://docs.pola.rs/user-guide/expressions/) for vectorized operations. This document explains how to extend the core with custom Rust functions.

## Architecture Overview

```text
┌─────────────────────────────────────────────────────────────┐
│  Python Layer (gaspatchio)                             │
│  ┌─────────────────────────────────────────────────────┐   │
│  │  Accessor Methods (.excel, .finance, .projection)    │   │
│  │  └─> Call register_plugin_function()                 │   │
│  └─────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────┘
                           │
                           ▼
┌─────────────────────────────────────────────────────────────┐
│  Polars Plugin System                                       │
│  └─> Routes to compiled Rust function via #[polars_expr]   │
└─────────────────────────────────────────────────────────────┘
                           │
                           ▼
┌─────────────────────────────────────────────────────────────┐
│  Rust Core (gaspatchio/core)                               │
│  ┌─────────────────────────────────────────────────────┐   │
│  │  Pure Rust functions operating on Polars Series      │   │
│  └─────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────┘
```

**Why Rust?**

- **Performance**: Vectorized operations on millions of rows
- **Memory efficiency**: Polars' streaming engine for bounded memory
- **Type safety**: Compile-time guarantees
- **Parallelism**: Automatic multi-threading via Polars

## Example: Excel IRR Function

The Excel `IRR()` function is implemented as a Rust plugin. Here's how it works:

### 1. Pure Rust Implementation

```rust
// gaspatchio-core/core/src/excel/irr.rs
use polars::prelude::*;

pub fn irr(inputs: &[Series], kwargs: &IrrKwargs) -> PolarsResult<Series> {
    let cash_flows = inputs[0].list()?;
    let guess = kwargs.guess;

    // Newton-Raphson iteration for IRR calculation
    let result: Float64Chunked = cash_flows
        .iter()
        .map(|opt_series| {
            opt_series.map(|s| calculate_irr(&s, guess))
        })
        .collect();

    Ok(result.into_series())
}
```

### 2. PyO3 Binding Layer

```rust
// gaspatchio-core/bindings/python/src/excel.rs
use pyo3_polars::derive::polars_expr;

#[polars_expr(output_type=Float64)]
pub fn irr(inputs: &[Series], kwargs: IrrKwargs) -> PolarsResult<Series> {
    gaspatchio_core_lib::excel_functions::irr(inputs, &kwargs)
}
```

### 3. Python Registration

```python
# docs-skip
# gaspatchio/accessors/excel_functions/irr.py
from polars.plugins import register_plugin_function
from gaspatchio import _internal

def irr(values: pl.Expr, guess: float = 0.1) -> pl.Expr:
    """Calculate Internal Rate of Return."""
    return register_plugin_function(
        args=[values],
        plugin_path=_internal.LIB,  # Path to compiled Rust library
        function_name="irr",
        kwargs={"guess": guess},
        is_elementwise=True,
    )
```

### 4. Accessor Integration

```python
# docs-skip
# gaspatchio/accessors/excel.py
from .excel_functions.irr import irr as _irr

@register_accessor("excel", kind="column")
class ExcelColumnAccessor(BaseColumnAccessor):

    def irr(self, guess: float = 0.1) -> "ExpressionProxy":
        """Calculate IRR for cash flow lists."""
        expr = _irr(self._get_polars_expr(), guess=guess)
        return ExpressionProxy(expr, self._get_parent_frame())
```

### 5. User-Facing API

```python
from gaspatchio import ActuarialFrame

af = ActuarialFrame({
    "policy_id": ["P001", "P002"],
    "cash_flows": [[-1000, 300, 400, 500], [-2000, 800, 900, 1000]]
})

# Clean, discoverable API
af.rate_of_return = af.cash_flows.excel.irr(guess=0.1)
```

## Core Functions Available

The Rust core provides these function categories:

| Category                | Functions                                                                               | Used By                                              |
| ----------------------- | --------------------------------------------------------------------------------------- | ---------------------------------------------------- |
| Excel                   | `irr`, `pv`, `yearfrac`                                                                 | Excel compatibility, financial calculations          |
| Vector / Polars plugins | `fill_series`, `accumulate`, `rollforward`, `list_pow`, `list_clip`, `list_conditional` | Projections, accumulation, account value rollforward |

## When to Use Rust vs Python

| Use Case                         | Recommendation                               |
| -------------------------------- | -------------------------------------------- |
| Element-wise on millions of rows | **Rust** - orders of magnitude faster        |
| Complex actuarial formulas       | **Rust** - performance critical              |
| One-off calculations             | **Python** `.apply()` is fine                |
| Prototyping                      | **Python** first, optimize to Rust later     |
| List operations (projections)    | **Rust** - `list_*` functions exist for this |

## Adding New Core Functions

Adding a new Rust function requires changes in three places:

### Step 1: Rust Core Function

```rust
// core/src/polars_functions/my_function.rs
use polars::prelude::*;

pub fn my_function(inputs: &[Series], kwargs: &MyKwargs) -> PolarsResult<Series> {
    // Implement logic using Polars primitives
}
```

### Step 2: PyO3 Binding

```rust
// bindings/python/src/vector.rs (or new file)
#[polars_expr(output_type_func = my_output_type)]
pub fn my_function(inputs: &[Series], kwargs: MyKwargs) -> PolarsResult<Series> {
    gaspatchio_core_lib::polars_functions::my_function(inputs, &kwargs)
}
```

### Step 3: Python Registration

```python
# docs-skip
# gaspatchio/functions/my_function.py
from polars.plugins import register_plugin_function

def my_function(expr: pl.Expr, **kwargs) -> pl.Expr:
    return register_plugin_function(
        args=[expr],
        plugin_path=LIB,
        function_name="my_function",
        kwargs=kwargs,
    )
```

## Testing Core Functions

```bash
# Rust unit tests
cd gaspatchio-core/core && cargo test

# Rust benchmarks
cd gaspatchio-core/core && cargo bench

# Python integration tests
cd gaspatchio-core/bindings/python && uv run pytest -v
```

## Summary

Gaspatchio's Rust core provides:

- **Polars plugin system** for registering high-performance functions
- **Type-safe Rust** implementation for actuarial calculations
- **Clean Python API** via accessors that hide the complexity
- **Extensibility** - add new functions following the established pattern

The Python accessor APIs (`.excel`, `.finance`, `.projection`) cover most modelling work. The Rust core is the layer underneath — reach for it when you want to push past the accessor surface for performance reasons or contribute new functions to the framework.

# Rollforward

## The Problem

Universal Life, Variable Annuity, and Indexed UL products share a structural challenge: charges at time *t* depend on the account value at *t*, which depends on those charges. Cost of Insurance is charged on `max(0, death_benefit - AV)` — but AV itself is moving as COI is deducted. Tiered fees depend on which AV band the policy sits in. IUL crediting clamps to a floor and cap applied to the running balance.

These dynamics can't be precomputed. Each period's calculation has to run in order, on the running balance, with earlier-period values feeding the later ones.

The rollforward API lets you write that within-period calculation once — as a chain of named steps — and runs it across every policy in your portfolio, period-by-period, while keeping a clear audit trail of what happened at each step.

Not every account-value calculation needs this

If your product's per-period cashflows can be precomputed before the loop (no charge depends on the running balance), use the simpler [`accumulate()`](https://gaspatchio.dev/0.9.0/concepts/projections/#escalation-accumulate-the-linear-recurrence) primitive — it's cheaper to write, just as fast, and right for term-life reserves, fixed-charge UL, and any pure linear recurrence.

## A Complete Example

```python
from datetime import date
import polars as pl
from gaspatchio import (
    ActuarialFrame,
    RollforwardBuilder,
    RollforwardCollector,
    Schedule,
    compile_rollforward,
)

af = ActuarialFrame(
    pl.DataFrame(
        {
            "av_init": [1_000.0, 5_000.0],
            "premium": [[100.0] * 12, [500.0] * 12],
            "coi_rate": [[0.001] * 12, [0.002] * 12],
            "sum_assured": [[50_000.0] * 12, [100_000.0] * 12],
            "admin_rate": [[0.01] * 12, [0.01] * 12],
            "interest_rate": [[0.004] * 12, [0.003] * 12],
        }
    )
)
sched = Schedule.from_calendar_grid(
    start_date=date(2025, 1, 31), n_periods=12, frequency="1M"
)
af = af.projection.set(schedule=sched)

b = af.projection.rollforward(
    states={"av": af["av_init"]},
)
(
    b["av"]
    .add(af["premium"], label="Premium")
    .deduct_nar(af["coi_rate"], death_benefit=af["sum_assured"], label="COI")
    .charge(af["admin_rate"], label="Admin")
    .grow(af["interest_rate"], label="Interest")
    .floor(value=0.0)
)

compiled = compile_rollforward(b)
collector = RollforwardCollector(compiled)
af.av = collector.expr_for("av")

result = af.collect()
print(result.select(["av_init", "av"]))
```

```text
shape: (2, 2)
┌─────────┬─────────────────────────────────┐
│ av_init ┆ av                              │
│ ---     ┆ ---                             │
│ f64     ┆ list[f64]                       │
╞═════════╪═════════════════════════════════╡
│ 1000.0  ┆ [1044.751356, 1089.276895, … 1… │
│ 5000.0  ┆ [5273.66367, 5545.946964, … 81… │
└─────────┴─────────────────────────────────┘
```

Terminal account values after twelve months: 1,522.36 and 8,194.37. The chain reads top-to-bottom as the within-period calculation order — each step operates on the *current* account value after all preceding steps. The `label` argument is how you refer back to a step later (see [Inspection](https://gaspatchio.dev/0.9.0/concepts/rollforward/inspection/index.md)).

## What You Can Do

| Capability          | What It Solves                                                                                 | Learn More                                                                                  |
| ------------------- | ---------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------- |
| **Step vocabulary** | Premium deposits, rate charges, COI on net amount at risk, IUL floor/cap, anniversary ratchets | [Choosing Steps](https://gaspatchio.dev/0.9.0/concepts/rollforward/steps/index.md)          |
| **Multi-account**   | VA + GMDB ratchet, GMWB run-off, secondary guarantees                                          | [Multi-State](https://gaspatchio.dev/0.9.0/concepts/rollforward/multi-state/index.md)       |
| **Variants**        | Build product variants from shared helper functions without duplicating code                   | [Building Variants](https://gaspatchio.dev/0.9.0/concepts/rollforward/composition/index.md) |
| **Inspection**      | Step-by-step rendering and fingerprinting for model governance                                 | [Inspection](https://gaspatchio.dev/0.9.0/concepts/rollforward/inspection/index.md)         |
| **Recipes**         | Ready-to-adapt within-period orderings for common products                                     | [Product Recipes](https://gaspatchio.dev/0.9.0/concepts/rollforward/products/index.md)      |

## Three Concepts to Know First

### Schedule

Every rollforward sits on an explicit projection — how many periods, what frequency, which calendar, which day-count. Declare it on the frame with `af.projection.set(...)` before calling `rollforward`. Anniversary masks, year-fractions, and period dates all derive from the same source and feed into the chain wherever they're needed.

```python
af = af.projection.set(
    valuation_date=date(2025, 1, 31),
    until="term_months",
    until_value=240,
    frequency="monthly",
)
```

For grids you want to share across frames or persist for audit, build a typed `Schedule` and pass it via `af.projection.set(schedule=sched)`. See [Schedules](https://gaspatchio.dev/0.9.0/concepts/schedules/index.md) for `from_inception`, business-day conventions, and joint-calendar support.

### Multiple Accounts

For a single-account product, write `b["av"]` and chain steps. For products with interacting accounts (a fund and a guarantee, an account value and a shadow), declare multiple accounts in `states={...}` and switch between them by indexing — `b["fund"]` for one, `b["guarantee"]` for the other. Steps execute in the order they're declared, regardless of which account they target.

### Compile, Then Collect

The builder is mutable while you're declaring steps. `compile_rollforward(b)` finalises it and runs validation. A `RollforwardCollector` then exposes the per-period account values as Polars expressions you assign onto an `ActuarialFrame`. Execution is deferred until `.collect()` — the projection runs once for all the values you ask for.

```python
compiled = compile_rollforward(b)
collector = RollforwardCollector(compiled)
af.av = collector.expr_for("av")  # eop value of av across all periods
result = af.collect()
```

By default the collector exposes each account's end-of-period value. To read a value *mid-chain* — between two steps within the period — declare a named point on the builder and target a step at it; that point then becomes retrievable with `collector.expr_for("av", point="post_charge")`. See [Multi-State](https://gaspatchio.dev/0.9.0/concepts/rollforward/multi-state/#custom-points-for-mid-chain-state) for the full pattern.

## Step Vocabulary at a Glance

| Family            | Methods                                                            | When to Use                                         |
| ----------------- | ------------------------------------------------------------------ | --------------------------------------------------- |
| **Absolute**      | `.add(expr)`, `.subtract(expr)`                                    | Premium deposits, flat-dollar fees, withdrawals     |
| **Rate**          | `.charge(rate)`, `.grow(rate)`, `.grow_capped(rate, floor=, cap=)` | M&E charges, interest, IUL crediting with floor/cap |
| **Actuarial**     | `.deduct_nar(rate, death_benefit=)`                                | COI on net amount at risk                           |
| **Multi-account** | `.ratchet(to=, when=)`                                             | GMDB high-water mark, secondary guarantee step-up   |
| **Bounds**        | `.floor(value=0.0)`                                                | Non-negative account-value clamp                    |

See [Choosing Steps](https://gaspatchio.dev/0.9.0/concepts/rollforward/steps/index.md) for the decision tree, formulas, and worked examples.

## Multi-State — When One Account Isn't Enough

For products with interacting accounts, declare both and switch between them by indexing:

```python
b = af.projection.rollforward(
    states={
        "fund": af["fund_init"],
        "gmdb": af["gmdb_init"],
    },
)
b["fund"].grow(af["rate"], label="Fund Return")
b["gmdb"].ratchet(
    to=pl.col("fund@eop"),
    when=af["anniv"],
    label="GMDB Ratchet",
)
```

The `pl.col("fund@eop")` notation reads the fund's end-of-period value within the same period — useful when the GMDB needs to ratchet to the post-growth fund balance, for example. No precomputed column is needed; the value is read live.

See [Multi-State](https://gaspatchio.dev/0.9.0/concepts/rollforward/multi-state/index.md) for VA + GMDB, GMWB run-off, and stop-condition patterns.

## Inspection and Governance

After compilation you can render the model as plain text suitable for audit reports, get a SHA-256 fingerprint that changes when the model structure changes, or read the structured form as a dict:

```python
compiled = compile_rollforward(b)
print(compiled.explain())        # human-readable summary
print(compiled.fingerprint())    # 'sha256:...'
print(compiled.canonical_form()) # machine-readable structure
```

See [Inspection](https://gaspatchio.dev/0.9.0/concepts/rollforward/inspection/index.md) for the full set of inspection helpers and common mistakes.

## What This API Does Not Do

- **No Python functions inside the projection.** Every step is declared up front; you can't drop in arbitrary Python mid-projection. This is what makes the model auditable, fingerprintable, and fast.
- **No automatic step reordering.** Steps run in the order you wrote them. Inspect the chain to verify the order matches your product spec.
- **No conditional step dispatch.** If different products need different chains, build each one explicitly — see [Building Variants](https://gaspatchio.dev/0.9.0/concepts/rollforward/composition/index.md). To turn a single step on or off per period, use `.ratchet(when=mask)` for ratchets or build a rate column that's zero where the step shouldn't fire.

## Runnable Examples

Three minimal patterns live in the source tree and are runnable as scripts:

- `gaspatchio/tutorials/rollforward-patterns/01_single_state_fund.py` — grow / charge / floor on one state (Hardy 2003 §6.3)
- `gaspatchio/tutorials/rollforward-patterns/02_multistate_ratchet.py` — fund + GMDB anniversary ratchet (Bauer/Kling/Russ 2008)
- `gaspatchio/tutorials/rollforward-patterns/03_lapse_stop.py` — withdrawal-driven termination via `lapse_when_all_non_positive` (Milevsky/Salisbury 2006)

Each script asserts internally against a closed-form expectation; running them is the success signal.

# Building Product Variants

## The Product Development Workflow

Real product development isn't linear. You build a base UL chain, then explore variations: "what if we add a rider charge?", "what if we strip the admin fee for the high-AV band?", "what if we replace flat crediting with an index strategy?".

The chain is mutable while you're building it. Each call to `b["av"].add(...)`, `.charge(...)`, etc. appends a step to the sequence. Once `compile_rollforward` runs, the sequence is fixed — variants are built by starting fresh builders and applying shared building blocks as ordinary Python.

The pattern: factor each block of related steps into a helper function that takes a builder and an account name. Compose the helpers in different orders for different product variants.

## Setup

Every example below shares one frame and schedule. Build a small two-policy UL frame on a two-year monthly grid, carrying every column the variants reference:

```python
from datetime import date

import polars as pl

from gaspatchio import (
    ActuarialFrame,
    RollforwardBuilder,
    RollforwardCollector,
    Schedule,
    compile_rollforward,
)

n_periods = 24
sched = Schedule.from_calendar_grid(
    start_date=date(2025, 1, 31), n_periods=n_periods, frequency="1M"
)
anniv_mask = sched.anniversary_mask()

af = ActuarialFrame(
    pl.DataFrame(
        {
            "policy_id": [1, 2],
            "av_init": [10_000.0, 25_000.0],
            "guarantee_init": [10_000.0, 25_000.0],
            "premium": [[1_200.0] * n_periods, [2_400.0] * n_periods],
            "coi_rate": [[0.0020] * n_periods] * 2,
            "sum_assured": [[250_000.0] * n_periods, [500_000.0] * n_periods],
            "admin_rate": [[0.0010] * n_periods] * 2,
            "interest_rate": [[0.0030] * n_periods] * 2,
            "fund_return": [[0.0050] * n_periods] * 2,
            "rider_rate": [[0.0005] * n_periods] * 2,
            "rider_rate_full": [[0.0005] * n_periods] * 2,
            "rider_active": [[True] * n_periods] * 2,
            "anniv_mask": [anniv_mask] * 2,
            "year_index": [list(range(1, n_periods + 1))] * 2,
        }
    )
)
af = af.projection.set(schedule=sched)

# Shared initial-value expression reused by several variant builders.
init = {"av": af["av_init"]}
```

## A Reusable Helper Pattern

```python
import polars as pl
from gaspatchio import ActuarialFrame, RollforwardBuilder, Schedule

def add_ul_charges(builder: RollforwardBuilder, state: str) -> None:
    """Standard UL within-period charges: COI, then admin."""
    h = builder[state]
    h.deduct_nar(
        pl.col("coi_rate"),
        death_benefit=pl.col("sum_assured"),
        label="COI",
    )
    h.charge(pl.col("admin_rate"), label="Admin")


def add_premium(builder: RollforwardBuilder, state: str) -> None:
    builder[state].add(pl.col("premium"), label="Premium")


def add_credit(builder: RollforwardBuilder, state: str) -> None:
    builder[state].grow(pl.col("interest_rate"), label="Interest")


def add_floor(builder: RollforwardBuilder, state: str) -> None:
    builder[state].floor(value=0.0)
```

Each helper appends a coherent block of steps. Now the variants compose:

```python
# All variants share the same projection — declare it once on the frame.
af = af.projection.set(schedule=sched)

# Vanilla UL
b_vanilla = af.projection.rollforward(states={"av": af["av_init"]})
add_premium(b_vanilla, "av")
add_ul_charges(b_vanilla, "av")
add_credit(b_vanilla, "av")
add_floor(b_vanilla, "av")

# UL with a rider (extra charge between Admin and Interest)
b_rider = af.projection.rollforward(states={"av": af["av_init"]})
add_premium(b_rider, "av")
add_ul_charges(b_rider, "av")
b_rider["av"].charge(af["rider_rate"], label="Rider")
add_credit(b_rider, "av")
add_floor(b_rider, "av")

# UL with no admin fee (e.g. a high-AV band)
def add_coi_only(builder: RollforwardBuilder, state: str) -> None:
    builder[state].deduct_nar(
        pl.col("coi_rate"),
        death_benefit=pl.col("sum_assured"),
        label="COI",
    )

b_no_admin = af.projection.rollforward(states={"av": af["av_init"]})
add_premium(b_no_admin, "av")
add_coi_only(b_no_admin, "av")
add_credit(b_no_admin, "av")
add_floor(b_no_admin, "av")
```

The ordering is just Python, so it's transparent and trivially testable. Use `compiled.explain()` after compile to verify the chain matches your spec for each variant.

## Conditional Inclusion

To include or exclude a step based on a configuration flag, use ordinary `if`:

```python
def build_variant(
    *,
    af: ActuarialFrame,
    has_rider: bool,
    has_admin: bool,
) -> RollforwardBuilder:
    b = af.projection.rollforward(states={"av": af["av_init"]})
    add_premium(b, "av")
    b["av"].deduct_nar(
        af["coi_rate"],
        death_benefit=af["sum_assured"],
        label="COI",
    )
    if has_admin:
        b["av"].charge(af["admin_rate"], label="Admin")
    if has_rider:
        b["av"].charge(af["rider_rate"], label="Rider")
    add_credit(b, "av")
    add_floor(b, "av")
    return b


b_full = build_variant(af=af, has_rider=True, has_admin=True)
b_no_rider = build_variant(af=af, has_rider=False, has_admin=True)
b_minimal = build_variant(af=af, has_rider=False, has_admin=False)
```

This is the recommended pattern for parameterising product flavours. Configuration lives in Python; the chain is determined unambiguously at build time; `compiled.fingerprint()` differs across variants and you have a paper trail of which combinations exist.

## Per-Period Conditional Application

The pattern above is **per-policy / per-build** — the step either runs every period or never. To gate a step on a per-period condition (e.g., a rider that only applies after year five), build the rate column with the condition baked in:

```python
b = af.projection.rollforward(states={"av": af["av_init"]})
add_premium(b, "av")
add_ul_charges(b, "av")

# Gate element-wise over the per-period list: the rider rate applies from year 5 on.
af.rider_rate_gated = pl.col("rider_rate_full") * pl.col("year_index").list.eval(
    (pl.element() >= 5).cast(pl.Float64)
)

b["av"].charge(af["rider_rate_gated"], label="Rider")

# 0.0 for years 1–4, then the full rider rate from year 5 on:
print(af.collect().get_column("rider_rate_gated").to_list()[0][:6])
```

A zero-rate `.charge` does nothing (multiplies by 1.0), so periods where the condition is `False` pass through unchanged. The same construction works for `.add`, `.subtract`, `.deduct_nar`, and `.grow` — produce a list-column expression that is zero (or one, for growth) when the step should sit out for a given period.

For `.ratchet`, the `when=` argument is already a per-period boolean indicator. Like other step args, it must be a single column reference — so compose any compound condition into a column on the frame first:

```python
b = af.projection.rollforward(
    states={"av": af["av_init"], "guarantee": af["guarantee_init"]}
)
b["av"].grow(af["fund_return"], label="Fund Return")

# Compose the per-period mask element-wise (both are list[bool] columns):
af.ratchet_mask = (
    pl.col("anniv_mask").cast(pl.List(pl.Int8))
    * pl.col("rider_active").cast(pl.List(pl.Int8))
).list.eval(pl.element() > 0)

b["guarantee"].ratchet(
    to=pl.col("av@eop"),
    when=af["ratchet_mask"],
    label="GMDB Ratchet",
)

# True only at anniversaries where the rider is active:
print(af.collect().get_column("ratchet_mask").to_list()[0])
```

## Sharing Schedules and Initial Values

Variants typically share the same projection and initial-value expression. Declare the projection on the frame, then pass the frame to each variant builder:

```python
def build_ul(
    *,
    af: ActuarialFrame,
    states_init: dict[str, pl.Expr],
    has_rider: bool = False,
) -> RollforwardBuilder:
    b = af.projection.rollforward(states=states_init)
    add_premium(b, "av")
    add_ul_charges(b, "av")
    if has_rider:
        b["av"].charge(af["rider_rate"], label="Rider")
    add_credit(b, "av")
    add_floor(b, "av")
    return b


# 20-year and 30-year variants share the same chain but different projections.
# Each variant lives on its own frame because the projection is part of the frame.
sched_20y = Schedule.from_calendar_grid(
    start_date=date(2025, 1, 31), n_periods=240, frequency="1M"
)
sched_30y = Schedule.from_calendar_grid(
    start_date=date(2025, 1, 31), n_periods=360, frequency="1M"
)
af_20 = af.projection.set(schedule=sched_20y)
af_30 = af.projection.set(schedule=sched_30y)
b_20 = build_ul(af=af_20, states_init={"av": af_20["av_init"]})
b_30 = build_ul(af=af_30, states_init={"av": af_30["av_init"]})
```

Different Schedules naturally produce different `compiled.fingerprint()` values — the time axis is part of the model's identity.

## Auditing Across Variants

Each compiled variant has its own fingerprint. Persist them alongside results:

```python
af = af.projection.set(schedule=sched)
variants = {
    "vanilla": build_ul(af=af, states_init=init, has_rider=False),
    "with_rider": build_ul(af=af, states_init=init, has_rider=True),
}
compiled = {name: compile_rollforward(b) for name, b in variants.items()}

for name, c in compiled.items():
    print(f"{name:<12}  {c.fingerprint()}")
```

```text
vanilla       sha256:5b5d5ab69eaa8bcd30c3269128c4899f37f565661d921380cd31594263b43f1f
with_rider    sha256:e48dbbe07a44a1ffae3b6e892fd66fa7b4d1c3576b334d9a718255f388200d7f
```

Distinct version stamps make structural changes self-evident. If a release said it would ship the `vanilla` variant but the production model's `fingerprint()` matches `with_rider`, the wrong product is in the valuation — and you see it before the AOM does.

If the value changes between two releases but the variant name didn't, the model structure drifted between them — a more reliable signal than diffing source files.

## What This Pattern Does Not Do

- **No mutate-after-compile.** `CompiledRollforward` is frozen. To change the chain you build a fresh builder, possibly using the same helpers, then call `compile_rollforward` again. Caching is your responsibility — typically by keying on the helper inputs.
- **No automatic step deduplication.** If two helpers both call `add_premium`, you get two Premium steps in the chain (and the premium gets added twice). Each step runs once per period; `compile_rollforward` does not collapse duplicates.
- **No structural diff helper.** Compare two variants by `print(c.canonical_form())` and diff the printed dicts.

# Step Cash Flows

> **The `track_increments` API surface exists, but per-step series are not emitted** — `RollforwardBuilder(..., track_increments=True)` and `RollforwardCollector.increment_for(label)` are stable, but calling `increment_for(...)` when the model runs fails with a `StructFieldNotFoundError`. This page describes the API shape and the supported way to attribute a single step today: difference two runs.

## The Problem: Where Did the Money Go?

IFRS 17 requires analysis of change — breaking down the movement in the contractual service margin, risk adjustment, and best-estimate liability into its component drivers. Profit testing requires understanding which charges generate margin and which are breakeven. Model validation requires reconciling the account value at each step.

Without increment tracking, the only way to isolate the impact of a single step is to run the model twice — once with the step enabled and once without — and difference the results. For a chain of five steps, that's six model runs to get a complete decomposition.

## The API Shape

Set `track_increments=True` on the builder, label every step, and request each cash flow by label from the collector. Per-step emission is not implemented, so `increment_for(...)` raises `StructFieldNotFoundError` rather than returning a series:

```python
# docs-skip
b = af.projection.rollforward(
    states={"av": af["av_init"]},
    track_increments=True,
)
b["av"].add(af["premium"], label="Premium")
b["av"].deduct_nar(
    af["coi_rate"],
    death_benefit=af["sum_assured"],
    label="COI",
)
b["av"].charge(af["admin_rate"], label="Admin")
b["av"].grow(af["interest_rate"], label="Interest")
b["av"].floor(value=0.0)

compiled = compile_rollforward(b)
collector = RollforwardCollector(compiled)

af.av = collector.expr_for("av")
af.premium_added = collector.increment_for("Premium")        # not emitted — raises
af.coi_charged = collector.increment_for("COI")              # not emitted — raises
af.admin_charged = collector.increment_for("Admin")          # not emitted — raises
af.interest_credited = collector.increment_for("Interest")   # not emitted — raises
```

Each cash flow is defined as the dollar change the corresponding step causes at each period:

- **Premium** = `+premium[t]` — added to AV
- **COI** = `−coi_rate[t] × max(0, SA − AV)` — deducted from AV
- **Admin** = `−admin_rate[t] × AV` — proportional fee reduces AV
- **Interest** = `+interest_rate[t] × AV` — growth increases AV

Positive values mean the step increased the account; negative values mean it decreased it.

## The Reconciliation Invariant

Per-step cash flows are defined so they sum exactly to the total change in account value over each period:

```python
# For every policy and every period t:
# av[t] − av[t−1] == Premium[t] + COI[t] + Admin[t] + Interest[t]
```

This is not an approximation — it is exact by construction. Each cash flow is the before-and-after balance difference for its step, so the telescoping sum equals the total change. That makes it suitable for regulatory reporting where reconciliation to the penny matters.

`.floor(value=0.0)` can break this invariant in periods where the account would otherwise have gone negative — the clamp absorbs the difference. To attribute that absorbed amount, reconstruct the pre-floor balance from the per-step cash flows and difference.

## Labels Are Required

When `track_increments=True` is set, every step that takes a label must have one. This is enforced at compile time:

```python
# docs-skip
b = af.projection.rollforward(states={"av": ...}, track_increments=True)
b["av"].add(af["premium"])   # ❌ no label — compile fails
```

```text
ValueError: track_increments=True requires every label-bearing Op to have label=...; Add has label=None
```

Labels are the addressing mechanism for cash flows; silently auto-generating them would make audit trails hard to reconcile across model versions. `.floor(value=...)` does not carry a label — it's a clamp, not a labelled cash flow.

## What Per-Step Cash Flows Are For

Per-step cash flows support:

- **IFRS 17 analysis of change** — split LRC/LIC movement into step-level drivers
- **Profit-test attribution** — allocate margin to charges and crediting
- **Model validation** — reconcile against an external reference model at each step
- **Sensitivity analysis** — read the COI cash flow directly without re-running with COI rates zeroed

To isolate a single step today, use the two-runs-and-difference approach: run the chain with and without the step and subtract.

# Inspection and Governance

## Seeing What Your Model Does

Before running a rollforward against 100,000 policies, you want to verify the step order matches the product spec. After running it, you want to confirm nothing has changed since last quarter.

The compiled rollforward exposes three inspection helpers — `explain()`, `fingerprint()`, and `canonical_form()` — that cover the typical governance, audit, and quarter-over-quarter change-detection flows.

## `compiled.explain()` — Human-Readable Summary

Renders the model as plain text suitable for audit reports, peer review, and TRACE logs. Building on the [Overview](https://gaspatchio.dev/0.9.0/concepts/rollforward/index.md) example:

```python
from datetime import date

import polars as pl

from gaspatchio import (
    ActuarialFrame,
    Schedule,
    compile_rollforward,
)

af = ActuarialFrame(
    pl.DataFrame(
        {
            "av_init": [1_000.0],
            "premium": [[100.0] * 12],
            "coi_rate": [[0.001] * 12],
            "sum_assured": [[50_000.0] * 12],
            "admin_rate": [[0.01] * 12],
            "interest_rate": [[0.004] * 12],
        }
    )
)
sched = Schedule.from_calendar_grid(
    start_date=date(2025, 1, 31), n_periods=12, frequency="1M"
)
af = af.projection.set(schedule=sched)

b = af.projection.rollforward(
    states={"av": af["av_init"]},
)
b["av"].add(af["premium"], label="Premium")
b["av"].deduct_nar(
    af["coi_rate"], death_benefit=af["sum_assured"], label="COI"
)
b["av"].charge(af["admin_rate"], label="Admin")
b["av"].grow(af["interest_rate"], label="Interest")
b["av"].floor(value=0.0)

compiled = compile_rollforward(b)
print(compiled.explain())
```

```text
Rollforward (spec_fingerprint = sha256:5b5d5ab69eaa8bcd30c3269128c4899f37f565661d921380cd31594263b43f1f)

States:
  av:  init=col("av_init")

Points:  bop, eop

Schedule: from_calendar_grid({'kind': 'from_calendar_grid', 'n_periods': 12, 'frequency': '1M', 'calendar': 'NullCalendar', 'convention': 'Unadjusted', 'day_count': 'OneTwelfth', 'anchor': 'month_end', 'start_date': '2025-01-31'})

Transitions (in order):
  av@eop  Add  [label='Premium']
  av@eop  DeductNAR  [label='COI']
  av@eop  Charge  [label='Admin']
  av@eop  Grow  [label='Interest']
  av@eop  Floor

batch_axes: ('policy',)
track_increments: False
lapse_when_all_non_positive: []
contract_boundary: None
engine_binding: portable
```

Use this to:

- Verify step order against the product specification before running the projection
- Document the model logic for peer review
- Capture model state in audit reports

The `engine_binding` line confirms how the projection will run. `portable` means every step uses only the supported set of operations and can run anywhere gaspatchio runs.

## `compiled.fingerprint()` — Change Detection

Returns a single value that pins down the model's structure — the steps, in their order, against their accounts, on their Schedule:

```python
compiled.fingerprint()
# 'sha256:a1b2c3d4e5f6...'
```

Two compiled rollforwards with the same step sequence in the same order, the same accounts, the same Schedule, and the same configuration produce the same value — even if the input column names differ. This means:

- Renaming `coi_rate` to `monthly_coi` doesn't change the value
- Changing a `label` from `"COI"` to `"Cost of Insurance"` does change it (labels are addressable identifiers, not aliases)
- Adding, removing, or reordering a step does change it

Use this for:

- **Model governance:** Record `fingerprint()` alongside each quarterly run. If it changes without a deliberate release, the model structure has drifted between quarters and your AOM is going to need to explain it.
- **Release evidence:** Compare values across releases to catch structural changes that slipped through review.
- **Audit trail:** Log the value in run metadata so you can show, for any historical result, which model actually produced it.

## `compiled.canonical_form()` — Machine-Readable Structure

Returns a structured description of the model — the same description that backs `fingerprint()`. Use it when you need to see *what* changed, not just *that* something did:

```python
import json
print(json.dumps(compiled.canonical_form(), indent=2))
```

```json
{
  "states": [
    {"name": "av", "init": "col(\"av_init\")"}
  ],
  "points": ["bop", "eop"],
  "transitions": [
    {"op": "Add", "target": "av@eop", "expr": "col(\"premium\")", "label": "Premium"},
    {"op": "DeductNAR", "target": "av@eop", "coi_rate": "col(\"coi_rate\")",
     "death_benefit": "col(\"sum_assured\")", "label": "COI"},
    {"op": "Charge", "target": "av@eop", "rate": "col(\"admin_rate\")", "label": "Admin"},
    {"op": "Grow", "target": "av@eop", "rate": "col(\"interest_rate\")", "label": "Interest"},
    {"op": "Floor", "target": "av@eop", "value": 0.0}
  ],
  "schedule": {
    "kind": "from_calendar_grid", "n_periods": 12, "frequency": "1M",
    "calendar": "NullCalendar", "convention": "Unadjusted",
    "day_count": "OneTwelfth", "anchor": "month_end",
    "start_date": "2025-01-31"
  },
  "track_increments": false,
  "lapse_when_all_non_positive": [],
  "contract_boundary": null,
  "engine_binding": "portable"
}
```

Diff two canonical forms to localise structural drift between releases.

## Direct Inspection of the Step Sequence

For programmatic introspection, walk the compiled chain directly:

```python
compiled.ir.transitions   # the steps, in order, as typed dataclasses
compiled.ir.states        # the declared accounts
compiled.ir.points        # ("bop", "eop") or any custom points
compiled.ir.schedule      # the bound Schedule
```

Each entry in `transitions` is a step record — `Add(target=..., expr=..., label=...)`, `Grow(...)`, `Charge(...)`, and so on. Use this when you need structured access for tooling — a CI check that asserts every COI step has a positive rate, a reporter that groups steps by family, or a structural diff between two model variants.

## Common Mistakes

### Using `.charge()` when you mean `.subtract()`

`.charge(rate)` multiplies: `state *= (1 - rate[t])`. It takes a *rate* (e.g., 0.01 for 1%).

`.subtract(expr)` subtracts a *dollar amount* (e.g., 15.00).

If the product spec says "$15 per month admin fee", use `.subtract`. If it says "0.15% of AV", use `.charge`.

### Forgetting `.floor(value=0.0)` on UL products

Without a floor, AV can go negative (and stay negative). Most UL contracts guarantee a non-negative account value at end of period. End the chain with `.floor(value=0.0)` unless the product spec explicitly allows negatives.

### Wrong step order

Steps execute in declaration order. Crediting interest before deducting COI means the policyholder earns interest on money that should have been charged. Match the order to the product's within-period calculation convention. Use `compiled.explain()` to verify.

### Annual rates without periodisation

If the projection is monthly but the rates are annual, periodise first:

```python
af.monthly_rate = af["annual_rate"] / 12             # simple
af.monthly_rate_compound = (1 + af["annual_rate"]) ** (1 / 12) - 1  # compound
```

The model uses what you pass it; it does not infer the calendar of the input rates. Convert annual rates to monthly (or whatever your projection frequency is) before assigning to the input frame.

### Cross-period account references

An account's value at period `t` is determined by the chain executed at `t`. There is no built-in way to read an account's value at `t−1` from inside the chain — sequencing periods *is* what the rollforward does for you. If you need a derivative quantity that depends on a prior-period value (a rolling-window cap on growth, for example), produce it as an input column with a `shift(1)` over an already-computed account output, then use it in a downstream rollforward.

### Schedule mismatch

For a **uniform** book, the Schedule's `n_periods` is the projection length, and every list-column input (`premium`, `coi_rate`, etc.) must have exactly that many elements per policy. If every policy's inputs instead share a *different* length, the projection raises a descriptive error rather than silently truncating to the wrong horizon — so set `n_periods` to match your inputs (or build the inputs to `n_periods`). For **jagged** books — variable per-policy horizons, either via `af.projection.set(..., per_policy=True)` or via input lists whose lengths differ across policies — each policy projects its own horizon from its own input-list length, and `n_periods` acts as a portfolio-max capacity hint. Either way, build the Schedule once and reuse it everywhere — for the inputs, for the chain, and for any anniversary masks.

# Multiple Accounts

## When One Account Isn't Enough

Some products track multiple accounts that interact during the projection:

- **VA + GMDB:** Account value and a guaranteed minimum death benefit that ratchets up when AV grows.
- **VA + GMWB:** Account value and a benefit base that reduces proportionally on withdrawal.
- **UL + secondary guarantee:** Account value and a shadow account that determines lapse eligibility.

These cannot be modelled as two independent rollforwards because the states reference each other within the same period — the guarantee needs to see the post-growth AV before it ratchets, the benefit base needs to see the pre-withdrawal AV to compute the proportional reduction.

## Setup

The short examples below share one single-policy frame on a twelve-month monthly grid, carrying every column they reference (the full worked example further down builds its own frame):

```python
from datetime import date

import polars as pl

from gaspatchio import (
    ActuarialFrame,
    RollforwardCollector,
    Schedule,
    compile_rollforward,
    when,
)

n_periods = 12
sched = Schedule.from_calendar_grid(
    start_date=date(2025, 1, 31), n_periods=n_periods, frequency="1M"
)
anniv_mask = sched.anniversary_mask()

af = ActuarialFrame(
    pl.DataFrame(
        {
            "policy_id": [1],
            "av_init": [100_000.0],
            "guarantee_init": [100_000.0],
            "fund_init": [100_000.0],
            "gmdb_init": [100_000.0],
            "premium": [[1_000.0] * n_periods],
            "me_rate": [[0.001] * n_periods],
            "fund_return": [[0.008] * n_periods],
            "rate": [[0.008] * n_periods],
            "anniv_mask": [anniv_mask],
        }
    )
)
af = af.projection.set(schedule=sched)
```

## Declaring Multiple Accounts

Multiple states are declared as entries in the `states` dict:

```python
b = af.projection.rollforward(
    states={
        "av": af["av_init"],
        "guarantee": af["guarantee_init"],
    },
)
```

Each entry's value is the per-policy initial-value expression — typically a column from the input frame. The schedule comes from the frame's projection (declared upstream with `af.projection.set(...)`). All accounts are tracked simultaneously and the steps run in declaration order across the chain.

## Routing Steps to an Account

Index the builder by account name to retrieve a handle. All step methods chain on the handle and append to a single shared sequence:

```python
b["av"].add(af["premium"], label="Premium")
b["av"].charge(af["me_rate"], label="M&E Fee")
b["av"].grow(af["fund_return"], label="Fund Return")
b["av"].floor(value=0.0)

b["guarantee"].ratchet(
    to=pl.col("av@eop"),
    when=af["anniv_mask"],
    label="GMDB Ratchet",
)
```

Steps execute in the exact order they were declared, regardless of which account they targeted. Two consecutive `b["av"].grow(...)` calls execute back-to-back; an interleaved `b["guarantee"].ratchet(...)` between them runs in the middle.

## Cross-Account Reads

The expression `pl.col("account@point")` reads the live value of another account at a specified point within the current period. This is how one account's calculation can depend on another account's same-period value.

```python
b = af.projection.rollforward(
    states={"fund": af["fund_init"], "gmdb": af["gmdb_init"]},
)
b["fund"].grow(af["rate"], label="Fund Return")
b["gmdb"].ratchet(
    to=pl.col("fund@eop"),       # reads fund's eop value, this period
    when=af["anniv_mask"],
    label="GMDB Ratchet",
)
```

Because `b["fund"].grow(...)` is declared first, the fund's eop value reflects the post-growth balance at the moment the ratchet evaluates. Reordering the declarations would change the semantics — the ratchet would see the pre-growth balance.

The point name must be one of the points declared on the builder. Default points are `("bop", "eop")`; custom points let you capture intermediate values mid-chain (see [Custom Points](#custom-points-for-mid-chain-state) below).

## Worked Example: VA with GMDB Ratchet

The canonical multi-state product: a variable annuity where the guaranteed minimum death benefit ratchets to the account value high-water mark on each anniversary. The full pattern below runs end-to-end:

```python
from datetime import date

import polars as pl

from gaspatchio import (
    ActuarialFrame,
    RollforwardCollector,
    Schedule,
    compile_rollforward,
    when,
)

# Five-year monthly schedule. Anniversary fires every 12 months.
n_periods = 60
sched = Schedule.from_calendar_grid(
    start_date=date(2025, 1, 31),
    n_periods=n_periods,
    frequency="1M",
)

# Schedule produces the anniversary mask: True at every contract anniversary.
anniv_mask = sched.anniversary_mask()  # list[bool] of length n_periods

af = ActuarialFrame(
    pl.DataFrame(
        {
            "av_init": [100_000.0],
            "guarantee_init": [100_000.0],
            "premium": [[1_000.0] * n_periods],
            "me_rate": [[0.001] * n_periods],
            "fund_return": [[0.008] * n_periods],
            "anniv_mask": [anniv_mask],
        }
    )
)
af = af.projection.set(schedule=sched)

b = af.projection.rollforward(
    states={
        "av": af["av_init"],
        "guarantee": af["guarantee_init"],
    },
)
b["av"].add(af["premium"], label="Premium")
b["av"].charge(af["me_rate"], label="M&E Fee")
b["av"].grow(af["fund_return"], label="Fund Return")
b["av"].floor(value=0.0)
b["guarantee"].ratchet(
    to=pl.col("av@eop"),
    when=af["anniv_mask"],
    label="GMDB Ratchet",
)

compiled = compile_rollforward(b)
collector = RollforwardCollector(compiled)
af.av = collector.expr_for("av")
af.guarantee = collector.expr_for("guarantee")
af.death_benefit = when(af.av > af.guarantee).then(af.av).otherwise(af.guarantee)

result = af.collect()
av = result.get_column("av").to_list()[0]
gtee = result.get_column("guarantee").to_list()[0]
db = result.get_column("death_benefit").to_list()[0]

print("Period  AV          Guarantee   Death Benefit")
for t in [0, 11, 12, 23, 24, 35, 36, 47, 48, 59]:
    print(f"  {t + 1:>2}  {av[t]:>12,.2f}  {gtee[t]:>12,.2f}  {db[t]:>12,.2f}")
```

```text
Period  AV          Guarantee   Death Benefit
   1    101,706.19    100,000.00    101,706.19
  12    121,280.31    121,280.31    121,280.31
  13    123,135.29    121,280.31    123,135.29
  24    144,416.40    144,416.40    144,416.40
  25    146,433.16    144,416.40    146,433.16
  36    169,570.13    169,570.13    169,570.13
  37    171,762.76    169,570.13    171,762.76
  48    196,917.44    196,917.44    196,917.44
  49    199,301.28    196,917.44    199,301.28
  60    226,649.63    226,649.63    226,649.63
```

What this output shows:

- Month 1: AV grows from 100,000 to 101,706 (premium + return − M&E fee, then floored). Guarantee unchanged at 100,000. Death benefit = max(AV, guarantee) = AV.
- Month 12 (first anniversary): the ratchet fires. Guarantee jumps to AV's eop value (121,280). Death benefit equals both.
- Month 13: AV continues to grow; guarantee holds at the locked-in 121,280 level.
- Pattern repeats every 12 months — guarantee steps up to the high-water mark, then holds until the next anniversary.

The pattern is also available as a runnable script with closed-form assertions: `bindings/python/gaspatchio/tutorials/rollforward-patterns/02_multistate_ratchet.py`. That script asserts the fund grows geometrically and the GMDB matches the fund value at every anniversary.

## Custom Points for Mid-Chain State

The default points `("bop", "eop")` capture state at start and end of period. To read a state's value *between* two steps in the chain — for example, to ratchet to AV after charges but before growth — declare an additional point and target it explicitly.

```python
b = af.projection.rollforward(
    states={
        "av": af["av_init"],
        "guarantee": af["guarantee_init"],
    },
    points=("bop", "post_charge", "eop"),
)

# Steps up to "post_charge" land on that point
b["av"].between("bop", "post_charge").charge(af["me_rate"], label="M&E Fee")

# Steps after default to "eop"
b["av"].grow(af["fund_return"], label="Fund Return")

# Guarantee reads AV at the post_charge point — pre-growth, post-fee
b["guarantee"].ratchet(
    to=pl.col("av@post_charge"),
    when=af["anniv_mask"],
    label="GMDB Ratchet",
)
```

Declared point order matters: `bop` must come first, `eop` last. Custom points sit between them and are filled as steps complete. Anything that should land on a custom point must use `.between(p1, p2)` to say so explicitly; by default a step targets `eop`.

## Coordinated Lapse Across Accounts

When a contract should terminate only if *all* tracked balances are exhausted simultaneously (e.g., a fund + secondary-guarantee shadow account), name them in `lapse_when_all_non_positive`:

```python
b = af.projection.rollforward(
    states={"av": af["av_init"], "shadow": af["shadow_init"]},
    lapse_when_all_non_positive=("av", "shadow"),
)
```

The lapse fires at end-of-period when *every* named state is ≤ 0. From that period onwards, both states write zero. States not listed in the tuple do not participate in the lapse check; they continue normally.

For a single-state termination pattern (the usual GMWB run-off), see `bindings/python/gaspatchio/tutorials/rollforward-patterns/03_lapse_stop.py` — fully runnable with hand-computed assertions.

## Same-Period Arithmetic — What's Allowed

Step arguments (`to=`, `rate=`, `expr=`, `when=`) accept a *single* column reference — either `af["col"]` for an input column or `pl.col("av@eop")` for a cross-account read. Compound expressions like `af["withdrawal"] / pl.col("av@post_grow")` aren't evaluated inside the chain.

Where the actuarial spec needs a derived rate (for example, a GMWB proportional reduction `1 − withdrawal / av_pre_withdrawal`), compute the rate as a regular column before the rollforward, then feed the materialised column to `.charge(...)` or wherever it belongs. The arithmetic stays auditable in the Polars query plan and the rollforward chain stays focused on the within-period sequence.

## See Also

- [Choosing Steps](https://gaspatchio.dev/0.9.0/concepts/rollforward/steps/index.md) — full step reference and decision tree
- [Inspection](https://gaspatchio.dev/0.9.0/concepts/rollforward/inspection/index.md) — verifying the chain order matches your spec
- `bindings/python/gaspatchio/tutorials/rollforward-patterns/02_multistate_ratchet.py` — runnable Bauer/Kling/Russ (2008) GMDB reference
- `bindings/python/gaspatchio/tutorials/rollforward-patterns/03_lapse_stop.py` — runnable GMWB-style lapse stop pattern

# Product Recipes

Ready-to-adapt rollforward patterns for common life-insurance and annuity products. Each recipe shows the standard within-period calculation order. Adapt the column names to match your data.

All recipes assume a Schedule has been built and an `ActuarialFrame` carries the input columns. The setup below builds a small three-policy frame carrying every column the recipes reference, on a twelve-month monthly grid. Each recipe reuses this `af`:

```python
from datetime import date

import polars as pl

from gaspatchio import (
    ActuarialFrame,
    RollforwardCollector,
    Schedule,
    compile_rollforward,
)

n_periods = 12
sched = Schedule.from_calendar_grid(
    start_date=date(2025, 1, 31), n_periods=n_periods, frequency="1M"
)
anniv_mask = sched.anniversary_mask()  # list[bool], True at each anniversary

af = ActuarialFrame(
    pl.DataFrame(
        {
            "policy_id": [1, 2, 3],
            # Initial balances
            "av_init": [10_000.0, 25_000.0, 0.0],
            "guarantee_init": [10_000.0, 25_000.0, 0.0],
            "reserve_init": [0.0, 0.0, 0.0],
            # Cash flows (per-period list columns)
            "premium": [[1_200.0] * n_periods, [2_400.0] * n_periods, [600.0] * n_periods],
            "dividend": [[150.0] * n_periods] * 3,
            "withdrawal": [[1_500.0] * n_periods, [3_000.0] * n_periods, [800.0] * n_periods],
            "net_premium": [[900.0] * n_periods, [1_800.0] * n_periods, [450.0] * n_periods],
            "expected_claims": [[700.0] * n_periods, [1_400.0] * n_periods, [350.0] * n_periods],
            # Rates
            "admin_rate": [[0.0010] * n_periods] * 3,
            "me_rate": [[0.0012] * n_periods] * 3,
            "interest_rate": [[0.0030] * n_periods] * 3,
            "valuation_rate": [[0.0025] * n_periods] * 3,
            "fund_return": [[0.0050] * n_periods] * 3,
            "index_return": [[0.0080] * n_periods] * 3,
            "floor_rate": [[0.0] * n_periods] * 3,
            "cap_rate": [[0.0100] * n_periods] * 3,
            "coi_rate": [[0.0020] * n_periods] * 3,
            "insurance_charge_rate": [[0.0015] * n_periods] * 3,
            # Benefits and masks
            "sum_assured": [[250_000.0] * n_periods, [500_000.0] * n_periods, [100_000.0] * n_periods],
            "anniv_mask": [anniv_mask] * 3,
            "in_force_mask": [[True] * n_periods] * 3,
        }
    )
)
af = af.projection.set(schedule=sched)
```

## Whole Life

Premiums, admin charges, guaranteed interest. No net-amount-at-risk because COI is bundled into the premium structure.

```python
b = af.projection.rollforward(
    states={"av": af["av_init"]},
)
b["av"].add(af["premium"], label="Premium")
b["av"].charge(af["admin_rate"], label="Admin")
b["av"].grow(af["interest_rate"], label="Interest")
```

## Universal Life

The standard UL chain. COI is charged on the net amount at risk; AV is floored at zero at end of period.

```python
b = af.projection.rollforward(
    states={"av": af["av_init"]},
)
b["av"].add(af["premium"], label="Premium")
b["av"].deduct_nar(
    af["coi_rate"], death_benefit=af["sum_assured"], label="COI"
)
b["av"].charge(af["admin_rate"], label="Admin")
b["av"].grow(af["interest_rate"], label="Interest")
b["av"].floor(value=0.0)
```

Order matters: premium is added first (increasing AV and reducing NAR before COI is calculated), admin is charged after COI (so the fee applies to the post-COI balance), and interest credits the final balance.

## Indexed UL (IUL)

Same as UL but with a floor-and-cap on the crediting rate. The policyholder participates in index returns but is protected on the downside and capped on the upside.

```python
b = af.projection.rollforward(
    states={"av": af["av_init"]},
)
b["av"].add(af["premium"], label="Premium")
b["av"].deduct_nar(
    af["coi_rate"], death_benefit=af["sum_assured"], label="COI"
)
b["av"].charge(af["admin_rate"], label="Admin")
b["av"].grow_capped(
    af["index_return"],
    floor=af["floor_rate"],    # typically 0.0
    cap=af["cap_rate"],        # e.g. 0.12 annual
    label="Index Credit",
)
b["av"].floor(value=0.0)
```

The `floor` and `cap` are themselves Polars expressions, so they can vary by period or by policy if your product spec uses tiered crediting bands.

## Variable UL (VUL)

Like UL but with a mortality-and-expense (M&E) charge and fund-based returns instead of guaranteed interest.

```python
b = af.projection.rollforward(
    states={"av": af["av_init"]},
)
b["av"].add(af["premium"], label="Premium")
b["av"].deduct_nar(
    af["coi_rate"], death_benefit=af["sum_assured"], label="COI"
)
b["av"].charge(af["me_rate"], label="M&E Fee")
b["av"].charge(af["admin_rate"], label="Admin")
b["av"].grow(af["fund_return"], label="Fund Return")
b["av"].floor(value=0.0)
```

M&E is charged before the fund return so the fee reduces the base on which returns are calculated — matching how separate-account charges work in practice.

## Variable Annuity (Accumulation Phase)

VA accumulation with no COI (no death-benefit risk charge during accumulation). M&E and fund return only.

```python
b = af.projection.rollforward(
    states={"av": af["av_init"]},
)
b["av"].add(af["premium"], label="Premium")
b["av"].charge(af["me_rate"], label="M&E Fee")
b["av"].grow(af["fund_return"], label="Fund Return")
b["av"].floor(value=0.0)
```

For VA products with guaranteed benefits (GMDB, GMWB), see [Multiple Accounts](https://gaspatchio.dev/0.9.0/concepts/rollforward/multi-state/index.md).

## VA + GMDB Ratchet

VA with a guaranteed minimum death benefit that ratchets to the AV high-water mark on each anniversary.

```python
b = af.projection.rollforward(
    states={
        "av": af["av_init"],
        "guarantee": af["guarantee_init"],
    },
)
b["av"].add(af["premium"], label="Premium")
b["av"].charge(af["me_rate"], label="M&E Fee")
b["av"].grow(af["fund_return"], label="Fund Return")
b["av"].floor(value=0.0)
b["guarantee"].ratchet(
    to=pl.col("av@eop"),
    when=af["anniv_mask"],
    label="GMDB Ratchet",
)
```

The setup above built `anniv_mask` with `sched.anniversary_mask()` (a materialised `list[bool]`); `sched.anniversary_mask_expr()` is the lazy-expression equivalent if you'd rather assign the column with `af.anniv_mask = sched.anniversary_mask_expr()`.

## GMWB Run-Off

A withdrawal contract that terminates when the fund is exhausted. The lapse stop-condition zeroes subsequent periods.

```python
b = af.projection.rollforward(
    states={"av": af["av_init"]},
    lapse_when_all_non_positive=("av",),
)
b["av"].subtract(af["withdrawal"], label="Withdrawal")
b["av"].grow(af["fund_return"], label="Fund Return")
```

The lapse-period state value retains its (possibly negative) computed value; subsequent periods are zero. Pair with `.floor(value=0.0)` if you want both clamping and zeroing.

## Term Life

Term contracts typically have no account value — they accrue reserves rather than fund balances. If you need a rollforward for term reserves, treat the reserve as the state and apply the per-period change:

```python
b = af.projection.rollforward(
    states={"reserve": af["reserve_init"]},
    contract_boundary=af["in_force_mask"],
)
b["reserve"].add(af["net_premium"], label="Net Premium")
b["reserve"].subtract(af["expected_claims"], label="Expected Claims")
b["reserve"].grow(af["valuation_rate"], label="Interest")
```

The `contract_boundary` indicator zeroes periods past the term end-date — useful when the same model runs across mixed durations.

## Credit Life

Reducing-balance coverage where the benefit amount decreases with the loan balance. Simple charge-and-growth, no COI on NAR.

```python
b = af.projection.rollforward(
    states={"av": af["av_init"]},
)
b["av"].charge(af["insurance_charge_rate"], label="Insurance Charge")
b["av"].grow(af["interest_rate"], label="Interest")
```

## Participating Whole Life

Whole life with annual dividends added to the account value.

```python
b = af.projection.rollforward(
    states={"av": af["av_init"]},
)
b["av"].add(af["premium"], label="Premium")
b["av"].add(af["dividend"], label="Dividend")
b["av"].charge(af["admin_rate"], label="Admin")
b["av"].grow(af["interest_rate"], label="Interest")
```

## Pulling Out the Result

After defining the chain, compile and collect. Here we use the VA + GMDB chain from above (it carries both an `av` and a `guarantee` state) so the multi-state extraction below has something to read:

```python
b = af.projection.rollforward(
    states={
        "av": af["av_init"],
        "guarantee": af["guarantee_init"],
    },
)
b["av"].add(af["premium"], label="Premium")
b["av"].charge(af["me_rate"], label="M&E Fee")
b["av"].grow(af["fund_return"], label="Fund Return")
b["av"].floor(value=0.0)
b["guarantee"].ratchet(
    to=pl.col("av@eop"),
    when=af["anniv_mask"],
    label="GMDB Ratchet",
)

compiled = compile_rollforward(b)
collector = RollforwardCollector(compiled)
af.av = collector.expr_for("av")
result = af.collect()
```

For multi-state recipes, request each state explicitly:

```python
af.av = collector.expr_for("av")
af.guarantee = collector.expr_for("guarantee")
```

Per-step cash-flow extraction via `track_increments=True` is API-stable but the projection does not emit per-step series — see [Step Cash Flows](https://gaspatchio.dev/0.9.0/concepts/rollforward/increments/index.md). To isolate a single step's contribution, run the chain twice (once with the step, once without) and difference.

# Choosing the Right Step

## Worked Example: Three Steps in Sequence

The smallest non-trivial chain: a separate-account variable annuity where the account value grows with the fund return, has an M&E charge deducted, and is floored at zero. Three steps, one account, twelve monthly periods. Copy-paste runnable:

```python
from datetime import date

import polars as pl

from gaspatchio import (
    ActuarialFrame,
    RollforwardCollector,
    Schedule,
    compile_rollforward,
)

n_periods = 12
sched = Schedule.from_calendar_grid(
    start_date=date(2025, 1, 31),
    n_periods=n_periods,
    frequency="1M",
)

af = ActuarialFrame(
    pl.DataFrame(
        {
            "av_init": [100_000.0],
            "fund_return": [[0.01] * n_periods],     # 1% monthly
            "me_charge": [[0.0010] * n_periods],     # 0.10% monthly
        }
    )
)
af = af.projection.set(schedule=sched)

b = af.projection.rollforward(
    states={"av": af["av_init"]},
)
(
    b["av"]
    .grow(af["fund_return"], label="fund_return")
    .charge(af["me_charge"], label="me_charge")
    .floor(value=0.0)
)

compiled = compile_rollforward(b)
collector = RollforwardCollector(compiled)
af.av = collector.expr_for("av")
av = af.collect().get_column("av").to_list()[0]

period_factor = (1 + 0.01) * (1 - 0.0010)
print(f"Period factor:   {period_factor:>12.6f}")
print(f"After {n_periods}M:       {av[-1]:>12,.2f}")
print(f"Closed-form:     {100_000.0 * period_factor**n_periods:>12,.2f}")
```

```text
Period factor:       1.008990
After 12M:         111,337.73
Closed-form:       111,337.73
```

The chain reads top-to-bottom as the within-period calculation order: grow first, then charge, then clamp. Same chain, runnable as a script with assertions: `bindings/python/gaspatchio/tutorials/rollforward-patterns/01_single_state_fund.py` (Hardy 2003 §6.3).

The remainder of this page is a reference for picking the right step for each kind of within-period calculation.

## Decision Tree

The most common question: "I have a charge or credit — which method do I use?"

```text
Is the amount proportional to AV or absolute?

  Proportional (e.g., "0.15% of account value")
    Reducing AV   .charge(rate)
    Growing AV    .grow(rate)
    Rate clamped between a floor and cap (IUL)
                  .grow_capped(rate, floor=, cap=)

  Absolute (e.g., "$15 per month")
    Adding        .add(expr)
    Removing      .subtract(expr)

  Depends on AV and another variable (e.g., "rate * max(0, SA - AV)")
                  .deduct_nar(rate, death_benefit=)

  Anniversary high-water mark on another state
                  .ratchet(to=pl.col("other@eop"), when=mask)

  Bound the result
                  .floor(value=0.0)
```

## Step Reference

### Absolute

These add or remove a fixed amount, independent of the current account value.

| Method            | Formula         | Use When                          |
| ----------------- | --------------- | --------------------------------- |
| `.add(expr)`      | `av += expr[t]` | Premium deposits, bonus additions |
| `.subtract(expr)` | `av -= expr[t]` | Flat-dollar fees, withdrawals     |

```python
b = af.projection.rollforward(states={"av": af["av_init"]})
b["av"].add(af["premium"], label="Premium")
b["av"].subtract(af["admin_fee_dollar"], label="Admin Fee")
```

### Rate

These apply a percentage charge or credit relative to the current account value.

| Method                             | Formula                                        | Use When                          |
| ---------------------------------- | ---------------------------------------------- | --------------------------------- |
| `.charge(rate)`                    | `av *= (1 - rate[t])`                          | M&E charges, admin fee as % of AV |
| `.grow(rate)`                      | `av *= (1 + rate[t])`                          | Interest crediting, fund returns  |
| `.grow_capped(rate, floor=, cap=)` | `av *= (1 + clamp(rate[t], floor[t], cap[t]))` | IUL crediting with floor and cap  |

```python
# M&E charge then fund return on a separate-account VA
b["av"].charge(af["me_rate"], label="M&E Fee")
b["av"].grow(af["fund_return"], label="Fund Return")

# IUL with 0% floor and 12% annual cap
b["av"].grow_capped(
    af["index_return"],
    floor=af["floor_rate"],
    cap=af["cap_rate"],
    label="Index Credit",
)
```

`floor` and `cap` are themselves Polars expressions, so they can vary by period or by policy if your data carries them as list columns or scalar columns.

### Account-Value-Dependent

The step's amount depends on the current account value — the defining characteristic of products where rollforward earns its keep.

| Method                              | Formula                                         | Use When                  |
| ----------------------------------- | ----------------------------------------------- | ------------------------- |
| `.deduct_nar(rate, death_benefit=)` | `av -= rate[t] * max(0, death_benefit[t] - av)` | COI on net amount at risk |

The net amount at risk (NAR) is `max(0, death_benefit - AV)`. When AV is close to the death benefit, NAR shrinks and the COI charge drops. This feedback loop is why COI cannot be pre-computed.

```python
b["av"].add(af["premium"], label="Premium")
b["av"].deduct_nar(
    af["coi_rate"],
    death_benefit=af["sum_assured"],
    label="COI",
)
b["av"].grow(af["interest_rate"], label="Interest")
```

### Multi-Account

These read or modify a *different* account from within the chain. Available when you've declared more than one entry in `states={...}`.

| Method                         | Formula                                                   | Use When                                          |
| ------------------------------ | --------------------------------------------------------- | ------------------------------------------------- |
| `.ratchet(to=expr, when=mask)` | `account = max(account, expr[t]) if mask[t] else account` | GMDB high-water mark, secondary guarantee step-up |

The `to=` expression typically uses the cross-account read syntax `pl.col("other_account@eop")` to capture another account's same-period value. The `when=` indicator is a per-period boolean — `True` triggers the ratchet, `False` leaves the account unchanged.

```python
# A two-account builder: a fund that grows, and a GMDB that ratchets to it.
af.anniv_mask = pl.lit(sched.anniversary_mask())  # per-period boolean column
b_gmdb = af.projection.rollforward(
    states={"fund": af["av_init"], "gmdb": af["av_init"]},
)
b_gmdb["fund"].grow(af["fund_return"], label="Fund Return")
b_gmdb["gmdb"].ratchet(
    to=pl.col("fund@eop"),
    when=af["anniv_mask"],
    label="GMDB Ratchet",
)
```

Where `anniv_mask` is a per-period boolean column. The Schedule produces one for you — see [Schedules](https://gaspatchio.dev/0.9.0/concepts/schedules/index.md) for `anniversary_mask()` and `anniversary_mask_expr()`.

See [Multiple Accounts](https://gaspatchio.dev/0.9.0/concepts/rollforward/multi-state/index.md) for the full pattern catalogue.

### Bounds

| Method           | Formula               | Use When                              |
| ---------------- | --------------------- | ------------------------------------- |
| `.floor(value=)` | `av = max(av, value)` | Non-negative account-value constraint |

Most UL and VA contracts guarantee a non-negative account value at end of period. End the chain with `.floor(value=0.0)` unless the product spec says otherwise.

```python
b["av"].add(af["premium"], label="Premium")
b["av"].deduct_nar(af["coi_rate"], death_benefit=af["sum_assured"], label="COI")
b["av"].grow(af["interest_rate"], label="Interest")
b["av"].floor(value=0.0)
```

## Builder-Level Configuration

Some behaviour belongs on the builder constructor rather than as a chained step.

### `track_increments=True`

Reserves per-step cash-flow tracking. The builder accepts the flag and the compile pass enforces that every step that takes a label has one set, but the projection itself does not emit the per-period cash-flow series. See [Step Cash Flows](https://gaspatchio.dev/0.9.0/concepts/rollforward/increments/index.md).

```python
b = af.projection.rollforward(
    states={"av": af["av_init"]},
    track_increments=True,
)
```

### `lapse_when_all_non_positive=("av",)`

Names accounts that — when *all* go to zero or below at end-of-period — terminate the projection. Every subsequent period writes zero across every account.

```python
b = af.projection.rollforward(
    states={"av": af["av_init"]},
    lapse_when_all_non_positive=("av",),
)
```

The lapse-period account value itself is not clamped — if a subtraction overshoots into negatives, that negative value appears in the output for the lapse period. Only *subsequent* periods are zeroed. Pair with `.floor(value=0.0)` if you want both: clamp the lapse-period value AND zero subsequent periods.

For coordinated lapse across multiple accounts (e.g., a secondary guarantee that holds the contract alive while the fund is exhausted), name them all. The lapse fires only when every named account is non-positive simultaneously.

### `contract_boundary=expr`

A boolean `pl.Expr` indicator whose first `True` value marks the period the contract leaves force. From that period onwards every state writes zero. Use for hard policy-term limits — for example, a 20-year term with no extension, or a paid-up-by date.

```python
b = af.projection.rollforward(
    states={"av": af["av_init"]},
    contract_boundary=af["expired_mask"],
)
```

Worked example — fund grows at 1%/month for the first three periods, then the contract expires at period 4:

```python
from datetime import date
import polars as pl
from gaspatchio import (
    ActuarialFrame,
    RollforwardCollector,
    Schedule,
    compile_rollforward,
)

sched = Schedule.from_calendar_grid(
    start_date=date(2025, 1, 31), n_periods=6, frequency="1M"
)
af = ActuarialFrame(
    pl.DataFrame(
        {
            "init": [100.0],
            "rate": [[0.01] * 6],
            # True at t=3 → contract is out of force from period 4 onwards.
            "expired_mask": [[False, False, False, True, False, False]],
        }
    )
)
af = af.projection.set(schedule=sched)
b = af.projection.rollforward(
    states={"av": af["init"]},
    contract_boundary=af["expired_mask"],
)
b["av"].grow(af["rate"])
compiled = compile_rollforward(b)
collector = RollforwardCollector(compiled)
af.av = collector.expr_for("av")

print(af.collect().get_column("av").to_list())
```

```text
[[101.0, 102.01, 103.0301, 0.0, 0.0, 0.0]]
```

The boundary indicator must be passed as a single column reference (`af["name"]`) — the rollforward captures a single column slot, not a re-evaluated expression. To derive the indicator from policy data (per-policy term end-dates, age-based termination, etc.), materialize it onto the frame first, then reference that column in the rollforward call. The most common pattern is to use the `af.projection.contract_boundary()` accessor:

```python
af.expired_mask = af.projection.contract_boundary(end_date_column="term_end_date")
b = af.projection.rollforward(
    states={"av": af["av_init"]},
    contract_boundary=af["expired_mask"],
)
```

The intermediate column name (`expired_mask` here) is yours to pick — any non-conflicting name works. This also lets the same chain run across a heterogeneous portfolio: each policy's per-row indicator zeroes its own state at its own term end-date.

`contract_boundary` and `lapse_when_all_non_positive` both terminate the projection but for different reasons: the boundary is a *time-driven* hard cutoff (the policy term simply ended), the lapse is a *balance-driven* termination (the fund went to zero). They can be combined — whichever fires first wins.

## Step Order

Steps execute in declaration order within each period. The order matters: `.charge` then `.grow` is *not* the same as `.grow` then `.charge`. The convention for separate-account products is to apply charges first, then growth, then bound:

```python
b["av"].charge(af["me_rate"], label="M&E")  # 1. fee on opening balance
b["av"].grow(af["fund_return"], label="Return")  # 2. apply return on net
b["av"].floor(value=0.0)  # 3. clamp non-negativity
```

Some product specs reverse charge and growth — match what your spec says. If unsure, `compiled.explain()` prints the chain in declaration order so you can verify it against the spec ([Inspection](https://gaspatchio.dev/0.9.0/concepts/rollforward/inspection/index.md)).

See [Multiple Accounts](https://gaspatchio.dev/0.9.0/concepts/rollforward/multi-state/index.md) for full VA + GMDB and GMWB examples.

# Aggregating at Scale

A reserving or capital run rarely needs every policy's full cashflow array kept in memory. What the valuation report wants is the *portfolio* shape: total net cashflow each month, the mean reserve, the tail of the loss distribution. Yet the obvious way to get there — project the whole book, concatenate every output, then aggregate — holds millions of per-period arrays in memory at once and falls over with an out-of-memory error long before the numbers land.

`run_aggregated` inverts that. It runs your projection over the portfolio in memory-safe batches and folds each batch down to per-period aggregates as it goes, so the peak memory you pay is one batch of output — not the whole book. You hand it the same `model_fn` you'd run on a single frame, the model points, and the aggregates you want; you get portfolio and per-period summaries over millions of policies, plus the telemetry to prove what the run cost.

You reach for `run_aggregated` when the deliverable is a portfolio summary — total cashflow by month, mean reserve, a tail quantile — rather than a per-policy output file, and the book is large enough that holding every projection at once is the thing standing between you and an answer.

______________________________________________________________________

## A portfolio fold

A small term-life book, projected three months forward. Net cashflow each month is premium less the running claim cost; the aggregates are the portfolio total and mean per month, a per-month upper tail of net cashflow, and the total present value as a single scalar.

```python
import polars as pl
from gaspatchio import ActuarialFrame
from gaspatchio.scenarios import PeriodSum, PeriodMean, PeriodQuantile, Sum, run_aggregated


def project(af: ActuarialFrame) -> ActuarialFrame:
    """Term-life net cashflow: premium less the running claim cost, three months out."""
    months = 3
    af.net_cf = pl.concat_list(
        [pl.col("premium") - pl.col("claim_cost") * (t + 1) for t in range(months)]
    )
    af.pv = pl.col("premium")
    return af


model_points = pl.DataFrame({
    "policy_id": [1, 2, 3, 4],
    "premium": [1200.0, 2400.0, 600.0, 1800.0],
    "claim_cost": [50.0, 120.0, 20.0, 90.0],
})

result = run_aggregated(
    project,
    model_points,
    [
        PeriodSum("net_cf").alias("total_net_cf"),
        PeriodMean("net_cf").alias("mean_net_cf"),
        PeriodQuantile("net_cf", levels=(0.95,)).alias("p95_net_cf"),
        Sum("pv").alias("total_pv"),
    ],
    batch_size=2,
)

print(result.total_net_cf)   # [5720. 5440. 5160.]  — portfolio net cashflow per month
print(result.mean_net_cf)    # [1430. 1360. 1290.]  — mean per policy per month
print(result.total_pv)       # 6000.0               — portfolio total, one scalar
```

The portfolio is sliced into batches of two policies. Each batch runs `project`, its output is folded into the running aggregates, and the batch is released before the next one starts — so the four-policy total comes back without ever holding all four projections at once. At a real book size the policy count changes; the memory profile does not.

Inside `model_fn`, build the projection columns with native `pl.col(...)` and assign them onto the frame as attributes. The aggregator names — `PeriodSum("net_cf")` — refer to the columns your model produces.

______________________________________________________________________

## Per-period arrays vs. portfolio scalars

The aggregators come in two shapes, and the shape decides what the result attribute holds.

The `Period*` family folds across policies *within each period*, so each one returns an array indexed by projection month:

```python
print(result.total_net_cf)   # ndarray, one entry per month: [5720. 5440. 5160.]
print(result.mean_net_cf)    # ndarray: [1430. 1360. 1290.]
```

`PeriodQuantile` returns a dict keyed by the levels you asked for, each value a per-period array — so one call gives you the same tail at every month:

```python
print(result.p95_net_cf)     # {0.95: array([...per month...])}
print(result.p95_net_cf[0.95])
```

The scalar family folds across *both* policies and periods to a single number — a portfolio total, the natural shape for a present value:

```python
print(result.total_pv)       # 6000.0
```

Each aggregator surfaces on the result under the name you gave it with `.alias(...)`. The alias is mandatory — it is the attribute you read the answer back from.

______________________________________________________________________

## Tail risk, folded in one pass

The valuation tail — the conditional expectation beyond a quantile — is the same fold pattern. `PeriodCTE` carries a mergeable sketch through the batches, so the tail estimate is built incrementally without re-reading the book:

```python
from gaspatchio.scenarios import PeriodCTE

tail = run_aggregated(
    project,
    model_points,
    [
        PeriodSum("net_cf").alias("total_net_cf"),
        PeriodCTE("net_cf", level=0.95, direction="lower").alias("cte95_net_cf"),
    ],
    batch_size=2,
)

print(tail.total_net_cf)     # [5720. 5440. 5160.]
print(tail.cte95_net_cf)     # per-month lower-tail conditional expectation
```

`direction="lower"` reads the adverse tail of net cashflow — the months where the book does worst. Because the sketch merges across batches, the tail you get from a hundred batches is the tail you'd get from one, to the sketch's stated accuracy.

______________________________________________________________________

## Read the telemetry

Every result carries what the run cost, so the memory claim is auditable rather than asserted:

```python
print(result.n_policies)     # 4     — policies folded
print(result.n_periods)      # 3     — projection months
print(result.batch_size)     # 2     — policies per batch
print(result.wall_time_s)    # seconds of wall time
print(result.peak_rss_mb)    # peak resident memory, MB
```

`peak_rss_mb` is the number that justifies the batch fold: it is the high-water mark of the whole run, and it tracks one batch of output rather than the full book. Quote it alongside `n_policies` and you have shown that a portfolio of any size was summarised inside a fixed memory budget.

______________________________________________________________________

## Letting the batch size find itself

The example pins `batch_size=2` so the fold is visible. In a production run, leave it at the default:

```python
# docs-skip
result = run_aggregated(project, model_points, aggregations)  # batch_size="auto"
```

With `batch_size="auto"`, the batch is sized to a memory budget — the largest batch whose predicted peak fits within the available allowance, read from the container's cgroup limit where one is set. You declare the aggregates you want; the run picks the largest batch that stays inside the memory you actually have, and `result.batch_size` reports the size it settled on.

______________________________________________________________________

## Partitioning by segment

By default the fold collapses the whole book into one figure per period. To get a figure *per segment* instead — by region, by product, by any model-point column — partition the aggregator with `.over(...)`:

```python
by_region = run_aggregated(
    project,
    model_points.with_columns(pl.Series("region", ["A", "A", "B", "B"])),
    [PeriodSum("net_cf").alias("net_cf_by_region").over("region")],
    batch_size=2,
)

print(by_region.net_cf_by_region)   # one row per (region, period)
```

The result carries one row per partition key per period, still computed inside the same memory budget — the partition is folded as the run goes, not by holding the segments apart. `.over(...)` takes one or more keys.

One aggregator does not partition: `PeriodQuantile.over(...)` is not supported — a per-partition streaming quantile is a separate sketch. For a result per *scenario* — each with its own shocked assumptions — reach for [`for_each_scenario`](https://gaspatchio.dev/0.9.0/concepts/scenarios/scenariorun/index.md).

______________________________________________________________________

## When to reach for `run_aggregated`

| You want...                                                    | Use                                                                                              |
| -------------------------------------------------------------- | ------------------------------------------------------------------------------------------------ |
| Per-policy output for every model point, too large for memory  | [`run_to_parquet`](https://gaspatchio.dev/0.9.0/concepts/scenarios/streaming-and-spill/index.md) |
| Portfolio / per-period summaries over a large book             | `run_aggregated` with `Period*` + scalar aggregators                                             |
| Those summaries split by a model-point segment                 | `run_aggregated` with `.over(...)`                                                               |
| A result per *scenario*, each with its own shocked assumptions | [`for_each_scenario`](https://gaspatchio.dev/0.9.0/concepts/scenarios/scenariorun/index.md)      |
| Proof of the memory the run cost                               | `result.peak_rss_mb` alongside `result.n_policies`                                               |

# Aggregators

When you run a plan across many scenarios, you need numbers that summarise the run — total claims, the worst case, capital at the 99.5% tail, the spread across runs. The aggregator layer turns each scenario's projection into one of those figures, and combines figures across scenarios in a way that's mergeable, partitionable, and reproducible byte-for-byte.

**Aggregators compute as the run progresses.** Each aggregator keeps a small running summary as scenarios stream past — `Sum` carries a running total, `Max` carries the largest-so-far, `Mean` carries `(total, count)`. The alternative — collecting every per-scenario value into one array and summarising at the end — is what you'd reach for in a spreadsheet, but it doesn't survive into the millions of scenarios. Streaming aggregation keeps peak memory bounded regardless of run length, which is what lets the same aggregator definitions run three deterministic scenarios and ten thousand stochastic ones unchanged.

Every aggregator carries:

- **a column** to read from your model's per-scenario output, and
- **a within-scenario reduction** (sum by default — see `within=` later).

You combine aggregators into a tuple, pass them to `for_each_scenario` or `ScenarioRun`, and read the result back by `.alias()`.

______________________________________________________________________

## Compute totals

The most common scenario question — what's the total across every scenario, and how many scenarios contributed?

```python
import polars as pl
from gaspatchio.frame import ActuarialFrame
from gaspatchio.scenarios import Sum, Count, for_each_scenario


def policies() -> ActuarialFrame:
    return ActuarialFrame({
        "policy_id": [1, 2, 3, 4, 5, 6, 7, 8],
        "age":       [30, 31, 32, 33, 30, 31, 32, 33],
        "premium":   [100.0, 150.0, 200.0, 250.0, 300.0, 350.0, 400.0, 450.0],
    })


def claim_model(af, *, tables=None, drivers=None):
    return af.with_columns(af["premium"].alias("claim"))


result = for_each_scenario(
    policies(),
    scenarios=["BASE", "STRESS_A", "STRESS_B"],
    model_fn=claim_model,
    aggregations=(
        Sum("claim").alias("total_claims"),
        Count("claim").alias("scenario_count"),
    ),
)

print(result.aggregations["total_claims"])    # 6600.0
print(result.aggregations["scenario_count"])  # 3
```

Each policy contributes its premium as a claim. Within each scenario the claims sum to 2,200; across three scenarios that sums to 6,600. `Count` is the number of scenarios that contributed at least one value — 3.

Two patterns to notice. The aggregator's column lives on the aggregator (`Sum("claim")`) — not on a separate config. The alias is the key you read the result by (`result.aggregations["total_claims"]`).

______________________________________________________________________

## Find the worst case

When you're stressing assumptions, "which scenario produced the largest loss" is just as important as the size of the loss. `ArgMax` answers it — it returns the scenario_id, not the value.

```python
from gaspatchio.scenarios import Max, Min, ArgMax, for_each_scenario


def stressed_model(af, *, tables=None, drivers=None):
    return af.with_columns(
        pl.when(pl.col("scenario_id") == "BASE").then(pl.col("premium") * 1.0)
          .when(pl.col("scenario_id") == "STRESS_LOW").then(pl.col("premium") * 1.2)
          .when(pl.col("scenario_id") == "STRESS_MED").then(pl.col("premium") * 1.5)
          .otherwise(pl.col("premium") * 2.0)
          .alias("claim"),
    )


result = for_each_scenario(
    policies(),
    scenarios=["BASE", "STRESS_LOW", "STRESS_MED", "STRESS_HIGH"],
    model_fn=stressed_model,
    aggregations=(
        Max("claim").alias("worst_case_claim"),
        Min("claim").alias("best_case_claim"),
        ArgMax("claim").alias("worst_scenario"),
    ),
)

print(result.aggregations["worst_case_claim"])  # 4400.0
print(result.aggregations["best_case_claim"])   # 2200.0
print(result.aggregations["worst_scenario"])    # 'STRESS_HIGH'
```

`ArgMin` is the mirror — the scenario_id of the lowest-claim scenario. On a tie, the lexicographically smallest scenario_id wins, so the result is reproducible.

______________________________________________________________________

## Capital at the tail

The Solvency II SCR is a 99.5%-tail conditional expectation. The `CTE` aggregator computes it directly — it averages every value above the 99.5th percentile across scenarios.

```python
from gaspatchio.scenarios import CTE, Quantile, Median, QuantileRank


scenarios = [f"STOCH_{i:03d}" for i in range(200)]


def stochastic_model(af, *, tables=None, drivers=None):
    sid = af["scenario_id"]
    factor = sid.str.slice(-3).cast(pl.Float64) / 200.0 * 3.0 + 1.0
    return af.with_columns((af["premium"] * factor).alias("claim"))


result = for_each_scenario(
    policies(),
    scenarios=scenarios,
    model_fn=stochastic_model,
    aggregations=(
        CTE("claim", level=0.005, direction="upper").alias("scr"),
        Quantile("claim", levels=(0.50, 0.95, 0.995)).alias("quantiles"),
        Median("claim").alias("median_claim"),
        QuantileRank("claim", at=5000.0).alias("rank_at_5k"),
    ),
)

print(result.aggregations["scr"])            # 8750.02
print(result.aggregations["quantiles"])      # {0.5: 5467.08, 0.95: 8437.99, 0.995: 8733.47}
print(result.aggregations["median_claim"])   # 5467.08
print(result.aggregations["rank_at_5k"])     # 0.4264432...
```

The sign convention is worth pinning. For a positive-is-loss column (the actuarial convention), 99.5% SCR is `CTE(level=0.005, direction="upper")` — average of values **above** the `1 - level` quantile. If your column is positive-is-profit, use `direction="lower"`.

`Quantile(column, levels=(...))` returns a dict keyed by level so you can read several quantiles in one pass. `Median` is shorthand for `Quantile(column, levels=(0.5,))` returning a bare float. `QuantileRank(column, at=value)` is the inverse — what fraction of scenarios sit below that value.

These four are sketch-backed (DDSketch). The output is mergeable across batches and bit-stable across processes given the same data. See [Sketch precision and memory](#sketch-precision-and-memory) at the end of the page for the tradeoff knobs.

______________________________________________________________________

## Spread of outcomes

Run-to-run dispersion — mean, variance, standard deviation. These use Welford+Chan parallel merge, so the result is order-independent and bit-stable across batch sizes.

```python
from gaspatchio.scenarios import Mean, Std, Variance


result = for_each_scenario(
    policies(),
    scenarios=scenarios,
    model_fn=stochastic_model,
    aggregations=(
        Mean("claim").alias("mean_claim"),
        Std("claim").alias("std_claim"),
        Variance("claim").alias("var_claim"),
    ),
)

print(result.aggregations["mean_claim"])  # 5483.5
print(result.aggregations["std_claim"])   # 1910.01
print(result.aggregations["var_claim"])   # 3648150.0
```

______________________________________________________________________

## Per-period results — the `Period*` family

Everything above folds a run to a single portfolio figure: one SCR, one mean, one worst case. But a cashflow projection isn't one number — it's a number per projection period. You need the net cashflow at each month, the p95 reserve along the whole run-off, the tail of claims period by period. `Sum("pv")` collapses that timeline to a scalar; the `Period*` aggregators preserve it.

A `Period*` aggregator reads a **per-period list column** — one list per policy, holding that policy's value at each projection period — and folds it down the policy axis at every period, returning an **array with one value per period**. `PeriodSum("net_cf")` is the per-period portfolio total: period 0 sums every policy's period-0 cashflow, period 1 sums every policy's period-1 cashflow, and so on. They run through `run_aggregated` exactly the way scalar aggregators run through `for_each_scenario` — build the tuple, `.alias()` each one, read the result back by alias.

```python
import polars as pl
from gaspatchio.frame import ActuarialFrame
from gaspatchio.scenarios import (
    PeriodSum, PeriodMean, PeriodQuantile, PeriodCTE, run_aggregated,
)


def project(af, *, tables=None, drivers=None):
    months = 3
    return af.with_columns(
        pl.concat_list(
            [pl.col("premium") - pl.col("claim_cost") * (t + 1) for t in range(months)]
        ).alias("net_cf"),
    )


model_points = pl.DataFrame({
    "policy_id":  [1, 2, 3, 4],
    "premium":    [1200.0, 2400.0, 600.0, 1800.0],
    "claim_cost": [50.0, 120.0, 20.0, 90.0],
})

res = run_aggregated(
    project,
    model_points,
    [
        PeriodSum("net_cf").alias("total_net_cf"),
        PeriodMean("net_cf").alias("mean_net_cf"),
        PeriodQuantile("net_cf", levels=(0.95,)).alias("p95_net_cf"),
        PeriodCTE("net_cf", level=0.95, direction="upper").alias("cte95_net_cf"),
    ],
    batch_size=2,
)

print(res.total_net_cf)   # [5720. 5440. 5160.]  — one total per month
print(res.p95_net_cf)     # {0.95: array([2194.45, 2079.05, 1963.51])}
print(res.cte95_net_cf)   # [1472.47 1399.88 1327.38]
print(res.n_periods)      # 3
```

Each policy's `net_cf` is a three-period list — premium less a claim cost that grows each month. `PeriodSum` adds those lists position by position: 1,150 + 2,280 + 580 + 1,710 = 5,720 in period 0, and so on down the run. `res.total_net_cf` is the portfolio cashflow at each period; `res.n_periods` reports how many periods the run produced. Read each alias straight off the result (`res.total_net_cf`) — same as `result.aggregations["..."]` for the scalar family.

`model_points` is a plain `pl.DataFrame` of policy data; `run_aggregated` slices it into batches of `batch_size` policies, hands each batch to your `model_fn` as an `ActuarialFrame`, and folds the per-period output of every batch into the same running summary. The aggregate is identical whether you run one policy per batch or all of them at once — `PeriodMean` carries exactly-additive `(sum, count)` state, so a per-period mean is batch-size-invariant rather than an average-of-batch-averages.

### The family

Each scalar aggregator has a `Period*` counterpart that returns a per-period array instead of a scalar:

| Aggregator                                                                  | Per-period result                                    |
| --------------------------------------------------------------------------- | ---------------------------------------------------- |
| `PeriodSum(column, within="sum")`                                           | portfolio total at each period                       |
| `PeriodMean(column)`                                                        | mean across the portfolio at each period             |
| `PeriodMin` / `PeriodMax`                                                   | extremes at each period                              |
| `PeriodStd` / `PeriodVariance`                                              | dispersion at each period                            |
| `PeriodCount(column)`                                                       | contributing policies at each period                 |
| `PeriodMedian(column, relative_accuracy=1e-4)`                              | median at each period                                |
| `PeriodQuantile(column, levels=(0.5,), relative_accuracy=1e-4)`             | `{level: per-period array}` for each requested level |
| `PeriodCTE(column, level=0.005, direction="upper", relative_accuracy=1e-4)` | tail average beyond `level` at each period           |

`PeriodQuantile` mirrors the scalar `Quantile`: `levels` is a tuple, and the result is a dict keyed by level, each value a per-period array — `res.p95_net_cf` is `{0.95: array([...])}`, one quantile estimate per period. `PeriodMedian` is the single-level shorthand. `PeriodCTE` carries the same sign convention as `CTE`: for a positive-is-loss column, the 99.5% tail is `level=0.005, direction="upper"` — the average of the values beyond the `1 - level` quantile, computed independently at every period. Flip to `direction="lower"` for a positive-is-profit column.

### Tail metrics at portfolio scale

`PeriodMedian`, `PeriodQuantile`, and `PeriodCTE` back their estimates with [**DDSketch**](https://www.vldb.org/pvldb/vol12/p2195-masson.pdf) — a peer-reviewed, fully-mergeable quantile sketch (Masson, Rim & Lee, VLDB 2019), the same one behind the scalar quantile aggregators, run independently for each projection period. A sketch is a fixed-memory structure that merges exactly across batches, so a per-period p95 over millions of policies resolves in one streaming pass with a bounded *relative* error (default `relative_accuracy=1e-4`, ~0.01%). That bounded-memory, exactly-mergeable property is what makes per-period tail metrics tractable at portfolio scale — you get a p95 reserve at every month of a run-off without ever holding the full per-policy distribution in memory. The precision/memory knob is the same `relative_accuracy` documented in [Sketch precision and memory](#sketch-precision-and-memory).

The DDSketch estimate is exact to its relative bound, not to the last digit: `PeriodMedian("net_cf")` on the data above returns `1430.03` against an exact `1430.00` — well inside the 0.01% tolerance, and identical regardless of `batch_size`.

When a run is long enough to watch, you can read these aggregates off each batch as it folds — [watching a tail metric converge](https://gaspatchio.dev/0.9.0/concepts/scenarios/streaming-convergence/index.md) before the run finishes.

______________________________________________________________________

## Modifiers

Every aggregator supports three modifiers. They're chainable in this order: `.alias(...)` first, then `.over(...)` or `.of(...)`.

### `.alias(name)` — required, names the output

```python
Sum("claim").alias("total_claims")
```

The alias is the key on `result.aggregations`. You must call `.alias()` on every aggregator passed to `for_each_scenario` or `ScenarioRun` — there is no implicit default. Aliases must be unique within a run.

### `.over(by)` — partitioned outputs

A partitioned aggregator returns a `pl.DataFrame` instead of a scalar, with one row per partition value.

```python
def lob_model(af, *, tables=None, drivers=None):
    return af.with_columns(
        pl.when(pl.col("policy_id") % 2 == 0).then(pl.lit("home"))
                                              .otherwise(pl.lit("motor"))
                                              .alias("lob"),
        af["premium"].alias("claim"),
    )


result = for_each_scenario(
    policies(),
    scenarios=["BASE", "STRESS"],
    model_fn=lob_model,
    aggregations=(
        Sum("claim").alias("total_claims"),
        Sum("claim").alias("by_lob").over("lob"),
    ),
)

print(result.aggregations["total_claims"])
# 4400.0

print(result.aggregations["by_lob"].sort("lob"))
# shape: (2, 2)
# ┌───────┬────────┐
# │ lob   ┆ by_lob │
# │ ---   ┆ ---    │
# │ str   ┆ f64    │
# ╞═══════╪════════╡
# │ home  ┆ 2400.0 │
# │ motor ┆ 2000.0 │
# └───────┴────────┘
```

The partition column must exist on the per-scenario projection your `model_fn` returns. Multi-key partitioning uses a tuple: `.over(("lob", "peril"))` produces a DataFrame keyed by both columns.

`ArgMax("claim").over("lob")` returns the worst scenario *per LOB*, which is a different question from `ArgMax("claim").alias("...")` returning the worst scenario overall.

The `Period*` (per-period) aggregators partition the same way. A partitioned per-period aggregator returns one row per partition *per projection period* — the per-period array is unfolded into `(partition, period, value)` rows:

```python
def cohort_model(af, *, tables=None, drivers=None):
    return af.with_columns(
        pl.when(pl.col("age") < 32).then(pl.lit("younger"))
                                   .otherwise(pl.lit("older")).alias("cohort"),
        pl.concat_list([pl.col("premium") - 10.0 * (t + 1) for t in range(3)]).alias("net_cf"),
    )


result = for_each_scenario(
    policies(),
    scenarios=["BASE", "STRESS"],
    model_fn=cohort_model,
    aggregations=(PeriodSum("net_cf").alias("nc_by_cohort").over("cohort"),),
)

print(result.aggregations["nc_by_cohort"].sort(["cohort", "period"]))
# shape: (6, 3)
# ┌─────────┬────────┬──────────────┐
# │ cohort  ┆ period ┆ nc_by_cohort │
# │ ---     ┆ ---    ┆ ---          │
# │ str     ┆ i64    ┆ f64          │
# ╞═════════╪════════╪══════════════╡
# │ older   ┆ 0      ┆ 2520.0       │
# │ older   ┆ 1      ┆ 2440.0       │
# │ older   ┆ 2      ┆ 2360.0       │
# │ younger ┆ 0      ┆ 1720.0       │
# │ younger ┆ 1      ┆ 1640.0       │
# │ younger ┆ 2      ┆ 1560.0       │
# └─────────┴────────┴──────────────┘
```

`PeriodQuantile` is the one aggregator whose `.over(...)` is not supported — its multi-level output has no single-column partitioned form. For a partitioned tail metric, reach for `PeriodMedian` or `PeriodCTE` with `.over(...)`; for an unpartitioned per-period quantile, use `PeriodQuantile` without `.over(...)`.

### `.of(pl.Expr)` — replace the within-scenario reduction

By default `Sum("claim")` sums the claim column within each scenario, then sums those scenario-level totals across scenarios. `.of(pl.Expr)` overrides the per-scenario reduction with a raw polars expression:

```python
# Per scenario: sum(claim × weight); then sum across scenarios.
weighted = Sum.of((pl.col("claim") * pl.col("weight")).sum()).alias("weighted_total")

# Per scenario: take the max claim; then take the mean across scenarios.
avg_worst = Mean.of(pl.col("claim").max()).alias("avg_worst_per_scenario")
```

`.of()` is bound to the polars backend — its expression is a `pl.Expr`. It does not survive YAML round-trip; reconstruct it in code when you reload a plan. For most analytic work `Sum("claim")` plus a `model_fn` that derives the column you want is the cleaner pattern.

______________________________________________________________________

## Within-scenario reductions

A second axis lives on every aggregator: how to reduce each scenario down to one value before aggregating across scenarios. Default is `"sum"`; the other named choices are `"mean"`, `"max"`, `"min"`, `"count"`, `"first"`, `"last"`.

```python
# Each scenario reduces to its max claim; then summed across scenarios.
Sum("claim", within="max").alias("sum_of_per_scenario_maxes")
```

`Sum("claim")` and `Sum("claim", within="sum")` are equivalent — that's the default.

______________________________________________________________________

## Sketch precision and memory

`Quantile`, `Median`, `CTE`, and `QuantileRank` — and their per-period counterparts `PeriodQuantile`, `PeriodMedian`, and `PeriodCTE` — are backed by DDSketch, a relative-error data structure that's exactly mergeable across batches and across processes. The per-period aggregators keep one sketch per projection period, so memory scales with the number of periods, not the number of policies.

The precision/memory knob is `relative_accuracy`. The defaults:

| Setting                            | Per-sketch memory | Tail precision at 99.5% |
| ---------------------------------- | ----------------- | ----------------------- |
| `relative_accuracy=1e-4` (default) | ~1.2 MB           | ~10 bp                  |
| `relative_accuracy=1e-3`           | ~125 KB           | ~100 bp                 |
| `relative_accuracy=1e-5`           | ~12 MB            | ~1 bp                   |

For SCR work, 10 bp at the 99.5% tail is well inside actuarial tolerance — you'd lose more than that to model assumption uncertainty. If you're aggregating many partitioned sketches (`.over(("region", "peril"))` with hundreds of partitions), tighten to `relative_accuracy=1e-3` to keep total memory bounded.

Two partition aggregators that share the same column don't share storage — each has its own sketch.

______________________________________________________________________

## Next

The aggregators on this page are the building blocks. The next page composes them into a [Scenario Run](https://gaspatchio.dev/0.9.0/concepts/scenarios/scenariorun/index.md) — a typed plan that captures shocks, base tables, and aggregations together, with a `source_sha` you can hand to model risk.

If you need an aggregator that's not built in (Skewness, Sharpe ratio, expected shortfall with custom weighting), see [Writing a custom aggregator](https://gaspatchio.dev/0.9.0/concepts/scenarios/custom-aggregators/index.md).

# Custom Aggregators

The 14 [built-in aggregators](https://gaspatchio.dev/0.9.0/concepts/scenarios/aggregators/index.md) cover most reporting needs, but you'll often want something that's not in the box — skewness for a tail-risk study, a portfolio Sharpe ratio, a weighted TVaR your regulator hasn't seen before. The aggregator layer is a plugin path: register your own class, use it in a `ScenarioRun` exactly like a built-in, and it survives YAML round-trip the same way.

You write five methods plus a canonical form. The framework handles batching, partitioning, merging across processes, and serialisation.

______________________________________________________________________

## When to write one

Reach for a custom aggregator when:

- The metric you need is mergeable across batches but isn't built in
- You want it to round-trip through YAML governance like the built-ins
- You're using it in more than one plan and want the metric to survive a code review

For a one-shot exploratory metric, the `.of(pl.Expr)` escape hatch on an existing aggregator is usually faster — see the modifiers section in the aggregators page.

______________________________________________________________________

## The contract

A custom aggregator implements five hooks plus a `canonical_form`. Inherit from `BaseAggregator` and you get the column, alias, `over()`, and `of()` modifiers for free.

```python
from dataclasses import dataclass
from typing import Any
from gaspatchio.scenarios import BaseAggregator, scenario_aggregator


@scenario_aggregator("Skewness")
@dataclass(frozen=True)
class Skewness(BaseAggregator):
    """Skewness across scenarios. Welford-Chan parallel-merge."""

    def create_accumulator(self) -> dict[str, float]:
        return {"n": 0.0, "mean": 0.0, "m2": 0.0, "m3": 0.0}

    def add_input(self, state, value):
        v = float(value) if value is not None else 0.0
        n1 = state["n"] + 1.0
        delta = v - state["mean"]
        delta_n = delta / n1
        term1 = delta * delta_n * state["n"]
        new_mean = state["mean"] + delta_n
        new_m3 = state["m3"] + term1 * delta_n * (n1 - 2.0) - 3.0 * delta_n * state["m2"]
        new_m2 = state["m2"] + term1
        return {"n": n1, "mean": new_mean, "m2": new_m2, "m3": new_m3}

    def merge_accumulators(self, a, b):
        na, nb = a["n"], b["n"]
        if na == 0:
            return b
        if nb == 0:
            return a
        n = na + nb
        delta = b["mean"] - a["mean"]
        mean = (na * a["mean"] + nb * b["mean"]) / n
        m2 = a["m2"] + b["m2"] + delta * delta * na * nb / n
        m3 = (
            a["m3"] + b["m3"]
            + delta ** 3 * na * nb * (na - nb) / (n * n)
            + 3.0 * delta * (na * b["m2"] - nb * a["m2"]) / n
        )
        return {"n": n, "mean": mean, "m2": m2, "m3": m3}

    def extract_output(self, state):
        n, m2, m3 = state["n"], state["m2"], state["m3"]
        if n < 3 or m2 == 0.0:
            return float("nan")
        std = (m2 / n) ** 0.5
        return (m3 / n) / (std ** 3)

    def canonical_form(self) -> dict[str, Any]:
        return {"kind": "Skewness", "column": self.column, "within": self.within}
```

The five hooks, in the order the framework calls them:

| Hook                 | Receives                          | Returns                   | Run when                       |
| -------------------- | --------------------------------- | ------------------------- | ------------------------------ |
| `create_accumulator` | nothing                           | a fresh accumulator state | once per scenario / partition  |
| `add_input`          | `state`, one `value` per scenario | new state                 | once per scenario contribution |
| `merge_accumulators` | two states                        | one merged state          | when batches merge             |
| `extract_output`     | final state                       | a JSON-serialisable value | once at the end                |
| `canonical_form`     | nothing                           | a recipe `dict`           | for SHA + YAML serialisation   |

The state can be any Python object — a dict, a tuple, a custom class. It travels in memory only, never to disk. What goes to disk is `canonical_form()` and `extract_output()`.

______________________________________________________________________

## Use it like a built-in

Once `@scenario_aggregator("Skewness")` runs, `Skewness("loss").alias("skew")` is just another aggregator.

```python
import polars as pl
from gaspatchio import ActuarialFrame
from gaspatchio.scenarios import Sum, for_each_scenario


def policies() -> ActuarialFrame:
    return ActuarialFrame({
        "policy_id": [1, 2, 3, 4, 5],
        "premium":   [1_000.0, 2_000.0, 1_500.0, 2_500.0, 3_000.0],
    })


def stressed(af, *, tables=None, drivers=None):
    sid = pl.col("scenario_id")
    factor = (
        pl.when(sid == "BASE").then(1.0)
          .when(sid == "MILD").then(1.5)
          .when(sid == "MEDIUM").then(2.0)
          .when(sid == "SEVERE").then(3.0)
          .otherwise(5.0)
    )
    return af.with_columns((af["premium"] * factor).alias("loss"))


result = for_each_scenario(
    policies(),
    scenarios=["BASE", "MILD", "MEDIUM", "SEVERE", "CATASTROPHIC"],
    model_fn=stressed,
    aggregations=(
        Sum("loss").alias("total"),
        Skewness("loss").alias("skew"),
    ),
)
print(result.aggregations["total"])  # 125000.0
print(result.aggregations["skew"])   # 0.80
```

A right-skewed stress distribution — the catastrophic scenario dominates the tail.

______________________________________________________________________

## Sketch-backed custom aggregators

Custom metrics that need a tail-quantile or a CTE can reuse the same `SignedSketch` the built-in `Quantile` / `CTE` use. The merge is bit-exact across batches and processes.

```python
from gaspatchio.scenarios._sketch import SignedSketch


@scenario_aggregator("TVaR95")
@dataclass(frozen=True)
class TVaR95(BaseAggregator):
    """Tail Value-at-Risk at 95% — DDSketch-backed mergeable."""

    relative_accuracy: float = 1e-4

    def create_accumulator(self) -> SignedSketch:
        return SignedSketch(relative_accuracy=self.relative_accuracy)

    def add_input(self, state, value):
        state.add(float(value))
        return state

    def merge_accumulators(self, a, b):
        return SignedSketch.merge(a, b)

    def extract_output(self, state):
        return state.cte(level=0.05, direction="upper")

    def canonical_form(self) -> dict[str, Any]:
        return {
            "kind": "TVaR95",
            "column": self.column,
            "within": self.within,
            "relative_accuracy": self.relative_accuracy,
        }
```

Validating against the built-in `CTE`:

```python
from gaspatchio.scenarios import CTE


def stochastic_model(af, *, tables=None, drivers=None):
    # One loss draw per (policy, scenario): the scenario index scales the
    # premium, giving a right-skewed loss distribution across the 200 draws.
    scale = 1.0 + pl.col("scenario_id").str.slice(1).cast(pl.Float64) / 100.0
    return af.with_columns((af["premium"] * scale).alias("loss"))


result = for_each_scenario(
    policies(),
    scenarios=[f"S{i:03d}" for i in range(200)],
    model_fn=stochastic_model,
    aggregations=(
        TVaR95("loss").alias("tvar_custom"),
        CTE("loss", level=0.05, direction="upper").alias("tvar_builtin"),
    ),
)
print(result.aggregations["tvar_custom"])    # 29402.20
print(result.aggregations["tvar_builtin"])   # 29402.20
print(result.aggregations["tvar_custom"] == result.aggregations["tvar_builtin"])
# True
```

Bit-exact match. Both use the same sketch with the same `relative_accuracy`; both add the same values in the same order; the final CTE call is the same. The custom path is identical to the built-in for this case — useful as a sanity check before you write something more exotic.

______________________________________________________________________

## Test the merge

The framework guarantees that:

> `extract(fold(A ++ B)) == extract(merge(fold(A), fold(B)))`

— for every aggregator, scalar or partitioned, across every batch boundary. Your custom aggregator must satisfy this for the run to be batch-equivalent. Pin it with a property test in your test suite:

```python
def fold(values, agg):
    state = agg.create_accumulator()
    for v in values:
        state = agg.add_input(state, v)
    return state


def test_skewness_merge_associative():
    agg = Skewness("x")
    A = [1.0, 2.0, 4.0, 7.0, 11.0]
    B = [16.0, 22.0, 29.0]

    extract_concat = agg.extract_output(fold(A + B, agg))
    extract_merge  = agg.extract_output(
        agg.merge_accumulators(fold(A, agg), fold(B, agg))
    )

    assert abs(extract_concat - extract_merge) < 1e-9
```

For sketch-backed aggregators, compare extract values rather than internal state — the merge is bit-exact at the bucket level but floating-point output can differ by 1 ULP if you compare states directly.

______________________________________________________________________

## YAML round-trip

`canonical_form()` is what gets serialised to YAML and rehydrated on reload. Two rules:

1. The `kind` field must match the name you registered with (`@scenario_aggregator("Skewness")`).
1. Every other field must match a constructor parameter name on your class.

The reload path is `cls(**{k: v for k, v in recipe.items() if k not in {"kind", "alias"}})`. If your `canonical_form()` emits `relative_accuracy=1e-4`, your constructor must accept `relative_accuracy` — which the `@dataclass` declaration handles automatically.

Custom aggregators that use the `.of(pl.Expr)` escape hatch do *not* survive YAML round-trip — the polars expression isn't serialisable into the recipe. If you reload such a plan, the framework raises a clear error rather than rebuilding with the wrong expression.

______________________________________________________________________

## Cross-process governance

For an audit handoff to work — counterparty saves YAML, you reload in a fresh interpreter and reproduce — your aggregator module must be imported in the fresh process *before* `ScenarioRun.from_yaml` runs. Otherwise the registry doesn't have a `Skewness` class to look up.

The current pattern is explicit: ship the module path with the YAML and have the auditor import it manually. A future YAML `plugins:` key will let plans self-describe their plugin dependencies; until then, document the imports alongside the plan.

______________________________________________________________________

## Cross-join semantics

When you write `add_input`, your aggregator receives **one value per scenario** — already reduced inside the scenario by `within_expr` (sum by default).

```python
def add_input(self, state, value):
    # `value` here is one number per scenario.
    # If you ran 100 scenarios across 1,000 policies, this is called 100 times
    # (or 100 × number-of-partitions, with .over).
    ...
```

What `add_input` does **not** receive is a row per policy. The per-scenario projection — a frame with one row per (policy_id, scenario_id) — is reduced inside the scenario by the `within` parameter (sum, mean, max, …) before it reaches your aggregator. This means:

- You don't write a Python loop over rows. The within-reduction is a polars expression and runs at speed.
- For an aggregator that genuinely needs per-policy data inside a single scenario (rare), use `Skewness.of(pl.col("loss"))` to override the within-reduction with a raw expression.

The framework cross-joins your `ActuarialFrame` with the batch's `scenario_id` column before `model_fn` runs, so every scenario sees every policy — that's where `model_fn` produces the columns your aggregator reads. The aggregator itself never sees individual policies.

______________________________________________________________________

## `requires_scenario_id` — when you need the scenario name

`ArgMin` / `ArgMax` are special: they need to remember *which* scenario produced the extreme value, not just the value. They opt in via a class attribute:

```python
# docs-skip
from typing import ClassVar


@scenario_aggregator("ArgWorstLossRatio")
@dataclass(frozen=True)
class ArgWorstLossRatio(BaseAggregator):
    """Scenario_id of the worst loss ratio observed."""

    requires_scenario_id: ClassVar[bool] = True

    def add_input(self, state, value):
        # `value` is now a (scenario_id, scalar) tuple instead of just scalar
        sid, ratio = value
        ...
```

With `requires_scenario_id = True`, the framework packs `(scenario_id, value)` into `add_input` instead of passing the bare value. Set it on classes that need to return a scenario identity rather than a scalar.

______________________________________________________________________

## Where this lands in the bigger story

A custom aggregator participates in everything you've seen on the previous pages: the `Sum` / `CTE` mix on the [aggregators](https://gaspatchio.dev/0.9.0/concepts/scenarios/aggregators/index.md) page, the [ScenarioRun](https://gaspatchio.dev/0.9.0/concepts/scenarios/scenariorun/index.md) plan layer, the audit chain, the YAML round-trip. There's no separate "custom path" — it's the same path the built-ins use.

# Natural Language → Executable Shock Configs

Gaspatchio's shock system is intentionally **LLM-friendly**: an LLM can take an English question and emit a JSON/dict config that is **parsed, validated, executed**, and then summarized back into English.

You do NOT generate new assumption tables again

Scenarios should be expressed as **small overlays** on top of your existing baseline assumptions.

- Your governed base tables stay untouched.
- Scenario configs are tiny, diffable artifacts.
- Shocks are applied **when the model runs** (or used to create shocked table copies in memory), not by exporting new “Scenario_123.xlsx” assumption tables.

______________________________________________________________________

## Why configs (not prose) are the point

When an actuary asks a question in English, the *model run* still needs deterministic inputs.

A JSON/dict config gives you:

- **Reproducibility**: rerun the exact scenario by reusing the same config.
- **Auditability**: store the config (or the equivalent `ScenarioRun.to_yaml()`) and the audit sidecar alongside results.
- **Composability**: combine simple “lego brick” operations (multiply/add/set/clip/max/min/pipeline) into regulatory scenarios.
- **No assumption-table churn**: configs describe *transformations*, not regenerated tables.

______________________________________________________________________

## The contract: what the LLM is allowed to output

At minimum, the LLM emits a **scenario config list**:

```json
[
  {"id": "BASE"},
  {
    "id": "SCENARIO_NAME",
    "shocks": [
      {"table": "mortality", "multiply": 1.2}
    ]
  }
]
```

Then Gaspatchio does the deterministic part:

```python
from gaspatchio.assumptions import Table
from gaspatchio.scenarios import ScenarioRun, Sum, parse_scenario_config

# The English question above lands as a scenario config list...
config = [
    {"id": "BASE"},
    {"id": "MORT_UP_20", "shocks": [{"table": "mortality", "multiply": 1.2}]},
]

# ...your governed base tables and the metrics you report on are already in hand.
base_tables = {
    "mortality": Table(
        name="mortality",
        source="assumptions/mortality.parquet",
        dimensions={"age": "age", "duration": "duration"},
        value="qx",
    ),
}
aggregations = (Sum("bel").alias("bel"),)

shocks = parse_scenario_config(config)
plan = ScenarioRun(shocks=shocks, base_tables=base_tables, aggregations=aggregations)
print(plan.describe())
```

That split matters: **the LLM proposes, the engine enforces** (schema + validation + execution).

______________________________________________________________________

## Actuarial prompts → configs that get executed

Below are examples of the kind of messy, dynamic English you actually get—and the configs an LLM can generate.

### 1) Duration-limited lapse stress (cohort + window)

**English**

> For TERM only, increase lapses by 25% but only in durations 1–3. Keep everything else base.

**Generated config**

```json
[
  {"id": "BASE"},
  {
    "id": "LAPSE_UP_25_DUR_1_3_TERM",
    "shocks": [
      {
        "table": "lapse",
        "multiply": 1.25,
        "where": {"product": "TERM", "duration": {"between": [1, 3]}}
      }
    ]
  }
]
```

**What executes**

- The base `lapse` table is **not replaced**.
- The lookup/transform applies `× 1.25` only where the filter matches.

______________________________________________________________________

### 2) Mass lapse at time 0 (classic regulatory shape)

**English**

> Add a 40% mass lapse at t=0 for UL, and cap lapse at 100%.

**Generated config**

```json
[
  {"id": "BASE"},
  {
    "id": "MASS_LAPSE_UL",
    "shocks": [
      {"table": "lapse", "add": 0.40, "when": {"t": {"eq": 0}}, "where": {"product": "UL"}, "clip": [null, 1.0]}
    ]
  }
]
```

______________________________________________________________________

### 3) “Solvency II lapse up” as a composable pipeline

**English**

> Solvency II lapse up: multiply by 1.5 but cap at 100%.

**Generated config**

```json
[
  {"id": "BASE"},
  {
    "id": "SII_LAPSE_UP",
    "shocks": [
      {
        "table": "lapse",
        "pipeline": [
          {"multiply": 1.5},
          {"clip": {"max": 1.0}}
        ]
      }
    ]
  }
]
```

______________________________________________________________________

### 4) Combined stress (IFRS17 / ORSA style)

**English**

> Worst-case combo: mortality +20%, expenses +10% (and cap negative expenses at 0), discount rates -100bps.

**Generated config**

```json
[
  {"id": "BASE"},
  {
    "id": "ADVERSE_COMBO",
    "shocks": [
      {"table": "mortality", "multiply": 1.2},
      {"table": "expense", "multiply": 1.1, "clip": [0.0, null]},
      {"table": "disc_rates", "add": -0.01}
    ]
  }
]
```

______________________________________________________________________

### 5) Table sensitivity sweep (actuarial “ladder” question)

**English**

> Give me PV impact for rates: -200, -100, -50, base, +50, +100 bps.

**Generated config**

```json
[
  {"id": "RATES_DOWN_200BPS", "shocks": [{"table": "disc_rates", "add": -0.02}]},
  {"id": "RATES_DOWN_100BPS", "shocks": [{"table": "disc_rates", "add": -0.01}]},
  {"id": "RATES_DOWN_50BPS", "shocks": [{"table": "disc_rates", "add": -0.005}]},
  {"id": "BASE"},
  {"id": "RATES_UP_50BPS", "shocks": [{"table": "disc_rates", "add": 0.005}]},
  {"id": "RATES_UP_100BPS", "shocks": [{"table": "disc_rates", "add": 0.01}]}
]
```

______________________________________________________________________

## Running the scenarios (no new tables; just overlays)

There are two common execution patterns:

### Pattern A: build shocked table copies in memory

This is the most literal “no new assumption tables” approach: you load base tables once, then create *derived* tables per scenario.

```python
import polars as pl
from gaspatchio.assumptions import Table
from gaspatchio.scenarios import parse_scenario_config

# 1) Load baseline tables once
mortality = Table(name="mortality", source="assumptions/mortality.parquet", dimensions={"age": "age", "duration": "duration"}, value="qx")
lapse = Table(name="lapse", source="assumptions/lapse.parquet", dimensions={"duration": "duration"}, value="rate")

# 2) Parse LLM-produced scenario config
# config is the list of scenario dicts emitted by the LLM (see JSON examples above)
config = [
    {"id": "BASE"},
    {
        "id": "MORT_UP_20_LAPSE_DOWN_20",
        "shocks": [
            {"table": "mortality", "multiply": 1.2},
            {"table": "lapse", "multiply": 0.8},
        ],
    },
]
scenarios = parse_scenario_config(config)  # dict[str, list[Shock]]

# 3) For each scenario, apply only relevant shocks to each base table
by_scenario = {}
for scenario_id, shocks in scenarios.items():
    mort_s = mortality
    lapse_s = lapse

    for s in shocks:
        if getattr(s, "table", None) == "mortality":
            mort_s = mort_s.with_shock(s)
        if getattr(s, "table", None) == "lapse":
            lapse_s = lapse_s.with_shock(s)

    by_scenario[scenario_id] = {"mortality": mort_s, "lapse": lapse_s}

# 4) Run your model using by_scenario[scenario_id]["mortality"].lookup(...)
```

### Pattern B: scenario-aware runs (vectorized) + scenario-varying assumptions

If your model is already using `with_scenarios()`, you can expand model points across scenario IDs and keep everything grouped by `scenario_id`. (This works especially well for large sweeps.)

See [Running Models Across Scenarios](https://gaspatchio.dev/0.9.0/concepts/scenarios/index.md).

______________________________________________________________________

## Turning results into English answers + charts

Gaspatchio gives you **scenario-indexed results** (usually a Polars DataFrame grouped by `scenario_id`).

From there, you typically:

1. Compute deltas vs `BASE` (PV, BEL, CSM, SCR/RBC metrics *that your model already computes*).
1. Render charts (bar charts for scenario ladders, waterfalls for attribution, time-series for surplus trajectories).
1. Have the LLM summarize the result table into an executive narrative.

A minimal sketch:

```python
import polars as pl

# `results` is your scenario-indexed model output — one row per
# (policy_id, scenario_id), carrying whatever metrics your model computed.
results = pl.DataFrame({
    "policy_id":  ["P001", "P002", "P003"] * 2,
    "scenario_id": ["BASE"] * 3 + ["ADVERSE_COMBO"] * 3,
    "pv_net_cf":  [1_240.0, 980.0, 1_510.0, 1_090.0, 820.0, 1_330.0],
})

summary = (
    results
    .group_by("scenario_id")
    .agg(pl.col("pv_net_cf").sum().alias("pv"))
)

base = summary.filter(pl.col("scenario_id") == "BASE").select("pv").item()
summary = summary.with_columns((pl.col("pv") - pl.lit(base)).alias("pv_delta"))

# LLM prompt input = summary.to_dicts() + the scenario config + (optional) plan.describe()
```

Note

Gaspatchio doesn't force a charting stack. In practice teams use Plotly/Matplotlib/Altair, then embed the resulting HTML/PNG into their report pipeline.

______________________________________________________________________

## Best practice: store the config, not a re-exported table

If you're doing anything audit/regulatory-adjacent, the artifact to keep is:

- the **scenario config** (JSON), or equivalently the `ScenarioRun` YAML
- the audit sidecar emitted by `plan.run(audit=True)` — carries `source_sha`, library versions, and every aggregator output
- the scenario-level output table(s)

That’s the whole point: **no regenerating assumption tables** for every scenario; just run baseline + overlays.

______________________________________________________________________

## See Also

- [What-If Analysis](https://gaspatchio.dev/0.9.0/concepts/scenarios/what-if/index.md) - the declarative config format
- [Shock Operations](https://gaspatchio.dev/0.9.0/concepts/scenarios/shocks/index.md) - full shock grammar (filters, pipeline, max/min, clip)
- [Table Sensitivities](https://gaspatchio.dev/0.9.0/concepts/scenarios/table-sensitivities/index.md) - apply shocks to existing tables in Python
- [Running Models Across Scenarios](https://gaspatchio.dev/0.9.0/concepts/scenarios/index.md) - vectorized scenario execution

# Performance at Scale

A run with 1,000 policies × 100 scenarios produces 100,000 rows; a stochastic study with 10,000 policies × 10,000 scenarios produces 100 million. Running those without a memory budget fails on a laptop. The scenario loop is built around three things that keep memory bounded regardless of run size: batched scenarios, mergeable aggregators, and an auto-sized batch.

This page covers when each knob matters and how to read the run-time signals.

______________________________________________________________________

## `batch_size` — the primary memory control

`for_each_scenario(..., batch_size=N)` runs at most `N` scenarios at a time. Aggregator state lives in memory between batches and merges across them; the per-batch projection (the cross-joined policy × scenario frame) is released as soon as the batch's reductions land.

```python
import polars as pl
from gaspatchio.frame import ActuarialFrame
from gaspatchio.scenarios import Sum, Mean, for_each_scenario


def policies():
    return ActuarialFrame({
        "policy_id": list(range(1, 101)),
        "premium": [100.0 + i for i in range(100)],
    })


def model(af, *, tables=None, drivers=None):
    return af.with_columns(af["premium"].alias("loss"))


scenarios = [f"S{i:04d}" for i in range(50)]

result = for_each_scenario(
    policies(),
    scenarios=scenarios,
    model_fn=model,
    aggregations=(
        Sum("loss").alias("total"),
        Mean("loss").alias("mean"),
    ),
    batch_size=4,
)

print(result.aggregations["total"])  # 747500.0
print(result.aggregations["mean"])   # 14950.0
print(result.batch_size)             # 4
print(result.batch_size_resolution)  # 'manual'
```

Peak in-memory size scales with `batch_size × policies × periods`, not with the full scenario count. A 10,000-scenario run at `batch_size=16` holds 16 scenarios worth of projection in memory at any time — the other 9,984 are scheduled batches or aggregator state.

______________________________________________________________________

## Bit-equivalence across batch sizes

Every built-in aggregator merges deterministically across batches. For most aggregators, the result is bit-identical regardless of `batch_size`:

```python
results = {}
for bs in (1, 4, 16):
    r = for_each_scenario(
        policies(), scenarios=scenarios, model_fn=model,
        aggregations=(Sum("loss").alias("total"),),
        batch_size=bs,
    )
    results[bs] = r.aggregations["total"]

print(results[1])   # 747500.0
print(results[4])   # 747500.0
print(results[16])  # 747500.0
```

**Bit-exact across `batch_size` (10 of 13 aggregators):**

| Aggregator                                  | Why it's bit-exact                                                                                     |
| ------------------------------------------- | ------------------------------------------------------------------------------------------------------ |
| `Sum`                                       | Neumaier-compensated summation -- error O(ε), order-stable                                             |
| `Count`, `Min`, `Max`, `ArgMin`, `ArgMax`   | Integer add or pick semantics -- exact                                                                 |
| `Quantile`, `Median`, `CTE`, `QuantileRank` | DDSketch buckets are integer counters; merge is commutative addition; quantile lookup is deterministic |

**Numerically stable but not bit-exact (`Mean`, `Variance`, `Std`):**

These three use the Welford-Chan online algorithm, whose parallel merge formula divides by `n_total` at intermediate steps. The result is associative *algebraically* but the floating-point rounding of the intermediate divisions is sensitive to how scenarios were split into batches. Drift is `O(ε · log N)` — for actuarial cashflow scales this is well below 1 ULP relative and well below any meaningful threshold for variance-of-loss work, but it isn't zero.

The choice is deliberate: Welford-Chan stays numerically stable when computing variance of large near-equal numbers (where the textbook `Σx² − (Σx)²/n` form loses all precision). Trading O(ε·log N) batch-size drift for that stability is the right call for actuarial use.

What this gets you in practice: pick `batch_size` to fit your memory budget. `Sum` and friends are auditably reproducible across batch sizes; `Mean`/`Variance`/`Std` are reproducible within their stated bound, which is far inside any actuarially meaningful precision.

The bit-equivalence guarantees here are pinned by `bindings/python/tests/scenarios/test_batch_equivalence.py`, which exercises every aggregator across `batch_size ∈ {1, 2, 4, 16, 64}`.

______________________________________________________________________

## `batch_size="auto"` — let the loop pick

If you don't want to tune by hand, `"auto"` probes memory at run time and picks a size targeting roughly half your available RAM:

```python
result = for_each_scenario(
    policies(),
    scenarios=scenarios,
    model_fn=model,
    aggregations=(Sum("loss").alias("total"),),
    batch_size="auto",
)

print(result.batch_size)             # 256 (or whatever the probe chose)
print(result.batch_size_resolution)  # 'auto_probe'
print(result.peak_rss_mb)            # 0.7 (delta over baseline)
```

`target_memory_fraction` (default 0.5) controls how aggressively the loop sizes itself. The probe runs **two** throwaway warm-up batches — one at `batch_size=1` and one at `batch_size=4` — measures the RSS delta of each, and fits a linear model `delta(size) = fixed_overhead + per_cell_cost · size`. The fixed term (base tables, encoder caches, Polars warmup) is paid once out of the memory budget; the remaining budget is divided by `per_cell_cost` to pick the run-time `batch_size`. The two-point fit is what stops the picker over-shooting on high-policy runs where fixed overhead dominates a single-point measurement. The result is stamped on `result.batch_size` so you can verify what it picked.

For most runs, `"auto"` is the right default. Override only when you have a specific reason — e.g., a CI runner with a 2 GB memory ceiling needs `batch_size=64` to stay under it deterministically.

______________________________________________________________________

## DDSketch memory

Most aggregators (`Sum`, `Count`, `Mean`, `Variance`) compute from running totals — each scenario adds to a small fixed accumulator and the answer doesn't depend on how many scenarios you've seen. `Quantile`, `Median`, `CTE`, and `QuantileRank` can't work that way. They need the *shape* of the distribution, not just running sums. The naive answer is to hold every value in memory and sort at the end; at 10 million scenarios × multiple partitions that's a lot of floats.

These four sidestep the problem with **DDSketch** — a streaming data structure that estimates quantiles to a *bounded relative error* using a *bounded amount of memory*, regardless of how many values you feed it. The bound is exact and mergeable: two sketches built from disjoint batches combine into one whose accuracy guarantee still holds. That's what lets the batched scenario loop produce stable, bit-equivalent quantile answers across batch sizes.

DDSketch is a published, peer-reviewed algorithm, not a homegrown approximation — [Masson, Rim & Lee, *DDSketch: A Fast and Fully-Mergeable Quantile Sketch with Relative-Error Guarantees*, VLDB 2019](https://www.vldb.org/pvldb/vol12/p2195-masson.pdf) ([doi:10.14778/3352063.3352135](https://doi.org/10.14778/3352063.3352135)). It is the same sketch Datadog runs across its metrics platform, and the relative-error and mergeability properties quoted here are the ones the paper proves. The quantile aggregators are backed by the authors' reference [`ddsketch`](https://pypi.org/project/ddsketch/) implementation directly, so what you reconcile against is the published algorithm rather than a re-derivation of it.

The tradeoff is exact quantiles for bounded ones. Two consequences for sizing:

- **Scaling with partitions.** A `CTE.over(("region", "peril"))` with 50 region/peril combinations holds 50 sketches per CTE, totalling ~60 MB just for that one aggregator. Multiple CTEs in the same plan multiply.
- **Bounded by accuracy choice.** Tightening to `relative_accuracy=1e-3` drops per-sketch memory to ~125 KB (10×) at the cost of widening tail error to ~100 bp (10×).

For Solvency II SCR work the default 10 bp tail error sits well inside actuarial assumption uncertainty. Tighten only if you genuinely need it; loosen if you're running many partitioned sketches and seeing memory pressure.

The scalar aggregators (Sum, Count, Min, Max, Mean, Variance, Std, ArgMin, ArgMax) carry trivial state (4 floats or fewer per partition slot). They aren't the variable that drives the memory curve.

______________________________________________________________________

## Scenario ID encoding

For runs with thousands of scenario IDs, encoding choice matters for the projection step (not the aggregator step):

```python
scenarios = list(range(10_000))                # int IDs — UInt32 under the hood
scenarios = [f"STOCH_{i:05d}" for i in range(10_000)]   # string IDs — slower
```

| ID type                    | Memory at 100M rows | Cross-join speed |
| -------------------------- | ------------------- | ---------------- |
| `int` (UInt32)             | ~400 MB             | Fastest          |
| Categorical-encoded string | ~400 MB             | Fast             |
| Plain string               | ~1.2 GB             | Slowest          |

For stochastic studies, integer scenario IDs are the cheapest. For a small named set (BASE / UP / DOWN), the readability of strings is worth the small cost.

______________________________________________________________________

## What to monitor

`ScenarioResult` carries three observability fields:

- `result.batch_size` — what the loop actually ran at
- `result.batch_size_resolution` — `'manual'`, `'auto_probe'`, or `'auto_calibrated'`
- `result.peak_rss_mb` — peak resident memory delta over baseline during the run
- `result.wall_time_s` — total wall time

If `peak_rss_mb` climbs unexpectedly between runs at the same `batch_size`, the projection has grown — usually because `model_fn` is producing wider frames than expected. Drop `batch_size` while you investigate.

______________________________________________________________________

## When the audit chain pins memory

[`ScenarioRun.run`](https://gaspatchio.dev/0.9.0/concepts/scenarios/scenariorun/index.md) carries the same `batch_size` argument. The recipe-level SHA does *not* include `batch_size` — two runs with different batch sizes share a `source_sha` because batch size is a memory choice, not a recipe choice. For bit-exact aggregators that means aggregator outputs are identical; for Welford-Chan aggregators (`Mean`/`Variance`/`Std`) outputs are reproducible within `O(ε·log N)`, far inside any actuarially meaningful precision.

If you write an audit sidecar, `run_metadata.batch_size` records what the run actually ran at, so an auditor reproducing the run can match the memory profile if they want — but they're not forced to.

# Scenario Run

A `ScenarioRun` is the plan you hand to model risk — shocks, base tables, aggregations, and an optional master seed bundled into a single object you can serialise to YAML, run, and reproduce byte-for-byte from the saved recipe alone.

Three things you get for the cost of building one:

- **A SHA over the recipe.** Two equivalent plans hash to the same `source_sha`; changing any shock parameter, table content, or aggregator changes it.
- **An opt-in audit sidecar.** A JSON file beside the run output that records what produced the numbers — the SHA, the canonical recipe, the library versions, and every aggregator output.
- **A bit-exact reload.** Save the plan to YAML, hand both to a different machine, reload, run on the same inputs — every scalar aggregator matches the original to the bit.

You'll reach for `ScenarioRun` when the run needs to be reproducible months later. For one-shot exploration the raw `for_each_scenario` loop is fine — you can [watch a long run converge](https://gaspatchio.dev/0.9.0/concepts/scenarios/streaming-convergence/index.md) as it streams, and promote to a `ScenarioRun` when the analysis is settled.

Promoting doesn't cost you the live view. `plan.run(...)` accepts the same `progress` and `on_batch` arguments as the raw loop, and because streaming is a live observation channel it never changes the plan's `source_sha`, its canonical form, or its audit sidecar — a quiet run and a watched run hash identically. The result also reports `n_batches`, the number of batches folded; it is runtime telemetry, not part of the SHA, and under `batch_size="auto"` it includes the auto-search probe batches.

______________________________________________________________________

## Build a plan

A Solvency II SCR plan, sized small but shaped like a real one — three stress shocks against mortality and lapse tables, plus a mass-lapse override.

```python
import polars as pl
from gaspatchio.assumptions import Table
from gaspatchio.frame import ActuarialFrame
from gaspatchio.scenarios import (
    CTE, ArgMax, MultiplicativeShock, OverrideShock, ScenarioRun, Sum,
)


def policies() -> ActuarialFrame:
    return ActuarialFrame({
        "policy_id": list(range(1, 11)),
        "age":       [30, 31, 32, 33, 34, 30, 31, 32, 33, 34],
        "sum_insured": [100_000.0] * 10,
    })


mortality = Table(
    name="mortality",
    source=pl.DataFrame({
        "age":  [30, 31, 32, 33, 34],
        "rate": [0.001, 0.0012, 0.0015, 0.0018, 0.0022],
    }),
    dimensions={"age": "age"},
    value="rate",
)

lapse = Table(
    name="lapse",
    source=pl.DataFrame({
        "age":  [30, 31, 32, 33, 34],
        "rate": [0.06, 0.05, 0.05, 0.04, 0.04],
    }),
    dimensions={"age": "age"},
    value="rate",
)


def bel_model(af, *, tables, drivers=None):
    """Expected claims = sum_insured × mortality × (1 - lapse)."""
    m  = tables["mortality"]
    lp = tables["lapse"]
    qx = m.lookup(scenario_id=af["scenario_id"], age=af["age"])
    px = lp.lookup(scenario_id=af["scenario_id"], age=af["age"])
    return af.with_columns(
        (af["sum_insured"] * qx * (1.0 - px)).alias("bel")
    )


plan = ScenarioRun(
    shocks={
        "BASE":         [],
        "MORTALITY_UP": [MultiplicativeShock(factor=1.5, table="mortality")],
        "LAPSE_UP":     [MultiplicativeShock(factor=1.5, table="lapse")],
        "MASS_LAPSE":   [OverrideShock(value=0.4, table="lapse")],
    },
    base_tables={"mortality": mortality, "lapse": lapse},
    aggregations=(
        Sum("bel").alias("bel"),
        CTE("bel", level=0.005, direction="upper").alias("scr"),
        ArgMax("bel").alias("worst_scenario"),
    ),
    master_seed=42,
)

print(plan.source_sha())
# sha256:48d9d126e...
```

A few things to notice. The model function takes a `tables` dict keyed by table name — the loop hands you the *shocked* version, stacked across scenarios. Your model code calls `Table.lookup` with both `scenario_id` and the table's dimension columns; the loop has already done the stacking.

The aggregations tuple mixes a scalar `Sum`, a sketch-backed `CTE`, and an `ArgMax` that returns a scenario name. Each carries its own column (`Sum("bel")`) and its own alias (`.alias("bel")`).

The `master_seed` participates in the SHA — two otherwise identical plans with different seeds have different `source_sha` values.

______________________________________________________________________

## Run with an audit sidecar

`plan.run(...)` accepts an `audit` argument. Pass a `Path` to write the sidecar to that exact location; pass `True` to write to a default location under `./gaspatchio_audit/<run_id>.audit.json`; leave it `False` (the default) for no sidecar.

```python
from pathlib import Path

result = plan.run(
    policies(),
    bel_model,
    batch_size=1,
    audit=Path("./run.audit.json"),
)

print(result.aggregations["bel"])             # 6030.0
print(result.aggregations["scr"])             # 2198.20
print(result.aggregations["worst_scenario"])  # 'MORTALITY_UP'
print(result.audit_path)                      # PosixPath('run.audit.json')
```

`result.aggregations` is keyed by your aliases — scalars come back as floats, partitioned aggregators (`.over(...)`) come back as `pl.DataFrame`. `result.audit_path` is the file the loop wrote, or `None` if you didn't ask for one.

______________________________________________________________________

## Read the audit sidecar

The sidecar is plain JSON. The reader is a one-liner.

```python
from gaspatchio.scenarios._audit import read_audit

audit = read_audit(Path("./run.audit.json"))

print(sorted(audit.keys()))
# ['aggregator_outputs',
#  'input_data_fingerprint',
#  'plan_canonical_form',
#  'run_metadata',
#  'schema_version',
#  'source_sha']

print(audit["schema_version"])              # '1.0'
print(audit["source_sha"] == plan.source_sha())  # True
print(audit["aggregator_outputs"])
# {'bel': 6030.0, 'scr': 2198.20..., 'worst_scenario': 'MORTALITY_UP'}

print(sorted(audit["run_metadata"].keys()))
# ['batch_size', 'batch_size_resolution', 'ddsketch_version',
#  'library_version', 'master_seed', 'n_scenarios',
#  'polars_version', 'python_version', 'wall_time_s']
```

`source_sha` ties the sidecar to the plan; `plan_canonical_form` is the full recipe (shocks, base table fingerprints, aggregations); `run_metadata` records the library versions you ran against. Partitioned aggregator outputs are coerced to a list of row dictionaries so the file stays portable.

______________________________________________________________________

## Save and reload from YAML

`plan.to_yaml(path)` writes the recipe — shocks, aggregations, master seed — but not the base table data (tables travel separately). `ScenarioRun.from_yaml(path, base_tables=...)` reconstructs the plan; you provide the tables yourself.

```python
plan.to_yaml(Path("./plan.yaml"))

reloaded = ScenarioRun.from_yaml(
    Path("./plan.yaml"),
    base_tables={"mortality": mortality, "lapse": lapse},
)

print(reloaded.source_sha() == plan.source_sha())  # True

reloaded_result = reloaded.run(policies(), bel_model, batch_size=1)
print(reloaded_result.aggregations["bel"] == result.aggregations["bel"])  # True
print(reloaded_result.aggregations["scr"] == result.aggregations["scr"])  # True
```

The SHA survives because the canonical form is order-independent — insertion order on the `shocks` dict and the `aggregations` tuple doesn't affect the hash, only content does. The aggregator values reproduce bit-exactly because the same inputs produce the same sketch state, the same Welford accumulator state, and the same fold order.

If you load a plan that was written in the pre-aggregator format (a `dict` rather than a list under `aggregations:`), `from_yaml` raises a pointed `ValueError` directing you at the migration path.

### Cross-process reproduction — the governance journey

The point of `to_yaml` / `from_yaml` isn't single-process round-tripping. It's the model-risk workflow: an actuary builds a plan, runs it, sends the YAML + the JSON audit sidecar to model risk; model risk reloads the plan in a different process — different shell, different machine, different week — and re-runs against the same model points + the same base tables. The plan SHA matches; the aggregator values match byte-for-byte. The audit chain pins exactly which run produced which numbers.

The recipe ships only the shock list, the aggregator list, the master seed, and a content fingerprint for each base table. The base tables themselves travel separately (parquet files in a governed assumptions folder, lifelib output, anything you can re-hand). On the consumer side:

```python
# docs-skip
# In the reviewer's shell — `plan.yaml` arrived alongside the model
# release notes; assumption tables live in a versioned assumptions folder.

base_tables = {
    "mortality": Table(name="mortality", source="assumptions/q4_2026/mortality.parquet", ...),
    "lapse":     Table(name="lapse",     source="assumptions/q4_2026/lapse.parquet",     ...),
}

reloaded = ScenarioRun.from_yaml("plan.yaml", base_tables=base_tables)

# Verify the plan SHA matches what the release notes pinned. If this fails,
# something changed — either the YAML or one of the base tables.
assert reloaded.source_sha() == "sha256:48d9d126..."

# Run against the same model points the original run used (parquet stamped
# alongside the release).
mp = pl.read_parquet("model_points_q4_2026.parquet")
result = reloaded.run(ActuarialFrame(mp), model_fn, batch_size=1)

# The aggregator values are bit-for-bit identical to the original run.
assert result.aggregations["bel"] == "<<the value the release notes quoted>>"
```

Three things make this work:

1. **Plan SHA is content-only.** Insertion order on `shocks` doesn't affect it; dict permutations and aggregator order shuffles produce the same hash.
1. **Base table fingerprints are recorded in the canonical form.** The reloaded plan checks that the supplied tables produce the same fingerprints the original plan recorded. A swapped parquet (even with the same name) breaks the SHA.
1. **Aggregator state is reproducibility-stable.** Welford accumulators (`Mean`, `Std`, `Variance`) merge order-independently; DDSketch quantile sketches (`Quantile`, `CTE`, `Median`, `QuantileRank`) are mergeable in any order with bit-exact equivalence; trivial reductions (`Sum`, `Count`, `Min`, `Max`) are by construction order-independent.

If the recipe drifts between the producer and the consumer side — a different `master_seed`, a different shock list, a different base-table fingerprint — `source_sha()` changes and the assertion fires. The failure is loud, structured, and tells you exactly which input shifted.

Limitation — non-flat shocks

Today, `to_yaml`/`from_yaml` round-trips the flat shock composables (`MultiplicativeShock`, `AdditiveShock`, `OverrideShock`). The conditional/composing shocks (`FilteredShock`, `TimeConditionalShock`, `PipelineShock`) do not yet have YAML serialisation. For non-flat plans, governance leans on the plan SHA + the JSON audit sidecar (both still work) rather than the YAML recipe — and the producing code lives in version control alongside the release.

______________________________________________________________________

## `master_seed` and stochastic kernels

If your model uses random draws (e.g., `numpy.random.default_rng(...)`), set `master_seed` and read `drivers["rng_seed"]` inside `model_fn`:

```python
def stochastic_bel(af, *, tables, drivers):
    rng = numpy.random.default_rng(drivers["rng_seed"])
    # use rng inside the kernel
    ...
```

The loop derives a per-scenario seed via `sha256(f"gsp-100|{master_seed}|{scenario_id}")`, so the same `(master_seed, scenario_id)` pair always produces the same draws — bit-stable across machines and Python versions.

`master_seed` is wired to `model_fn` only at `batch_size=1`. With `batch_size > 1` the loop raises a clear `ValueError` rather than silently dropping the seed. The same constraint applies to the drivers-dict scenario shape.

______________________________________________________________________

## Composers

`ScenarioRun` is immutable. Three helpers return new plans:

```python
seeded   = plan.with_master_seed(42)
stressed = plan.with_extra_shocks(
    {"MORTALITY_DOWN": [MultiplicativeShock(factor=0.8, table="mortality")]}
)
broader  = plan.with_extra_aggregations(Sum("bel").alias("by_age").over("age"))
```

Each returns a new plan with a new `source_sha`. The original is unchanged.

______________________________________________________________________

## When to reach for `ScenarioRun`

| You want...                                     | Use                                           |
| ----------------------------------------------- | --------------------------------------------- |
| Quick exploration; one-shot stress              | `for_each_scenario(...)` raw                  |
| A run you'll reproduce in 6 months              | `ScenarioRun` + `audit=True`                  |
| A plan you'll hand to model risk or a regulator | `ScenarioRun` + `to_yaml` + `audit=Path(...)` |
| A run that needs to match across machines       | `ScenarioRun(..., master_seed=...)` + audit   |

______________________________________________________________________

## Next

You've now seen the full pre-built surface — 14 aggregators, the typed plan, the audit chain. If you need an aggregator that's not in the box, the next page walks through [writing a custom aggregator](https://gaspatchio.dev/0.9.0/concepts/scenarios/custom-aggregators/index.md).

# Shock Operations

Gaspatchio provides a composable shock system for actuarial stress testing. Think of shocks as **lego bricks** - simple operations you can combine to build complex regulatory scenarios.

This page teaches you:

1. **The building blocks** - Individual shock operations (multiply, add, clip, etc.)
1. **How to filter** - Apply shocks to specific segments or time periods
1. **How to compose** - Chain operations together for complex transformations
1. **How to build regulatory scenarios** - Solvency II SCR as a worked example

______________________________________________________________________

## The Building Blocks

Every shock transforms assumption values. The basic operations are:

| Operation  | What it does         | Example use case            |
| ---------- | -------------------- | --------------------------- |
| `multiply` | Scale by a factor    | "Increase mortality by 20%" |
| `add`      | Add a constant       | "Shift rates up 50bps"      |
| `set`      | Replace with a value | "Assume zero lapses"        |
| `clip`     | Cap/floor values     | "Lapse cannot exceed 100%"  |

### Multiplicative Shocks

Scale values by a factor. Use for percentage changes.

```python
from gaspatchio.scenarios import MultiplicativeShock

# Increase mortality by 20%
shock = MultiplicativeShock(factor=1.2, table="mortality")

# Decrease lapse rates by 10%
shock = MultiplicativeShock(factor=0.9, table="lapse")
```

**Config syntax:**

```json
{"table": "mortality", "multiply": 1.2}
```

### Additive Shocks

Add a constant delta. Use for basis point shifts and absolute changes.

```python
from gaspatchio.scenarios import AdditiveShock

# Add 50bps to discount rates
shock = AdditiveShock(delta=0.005, table="discount_rates")

# Subtract 1% from expense loading
shock = AdditiveShock(delta=-0.01, table="expenses")
```

**Config syntax:**

```json
{"table": "discount_rates", "add": 0.005}
```

### Override Shocks

Replace all values with a constant. Use for extreme scenarios and boundary testing.

```python
from gaspatchio.scenarios import OverrideShock

# Set lapse rates to zero
shock = OverrideShock(value=0.0, table="lapse")

# Override discount rate to flat 5%
shock = OverrideShock(value=0.05, table="discount_rates")
```

**Config syntax:**

```json
{"table": "lapse", "set": 0.0}
```

### Clip Shocks

Cap and/or floor values to a range. Essential for keeping shocked values actuarially valid.

```python
from gaspatchio.scenarios import ClipShock

# Cap lapse rates at 100%
shock = ClipShock(max_value=1.0, table="lapse")

# Floor mortality at 0.1%
shock = ClipShock(min_value=0.001, table="mortality")

# Clip to a valid range
shock = ClipShock(min_value=0.0, max_value=1.0, table="rates")
```

**Config syntax:**

```json
{"table": "lapse", "clip": {"max": 1.0}}
{"table": "mortality", "clip": {"min": 0.001}}
{"table": "rates", "clip": {"min": 0.0, "max": 1.0}}
```

**Syntactic sugar** - use `[min, max]` arrays with `null` for unbounded:

```json
{"table": "lapse", "clip": [null, 1.0]}
{"table": "rates", "clip": [0.0, 1.0]}
```

______________________________________________________________________

## Filtering Shocks

Shocks can be scoped to a subset of rows — specific product segments, time windows, or any condition expressible against the table.

### WHERE Clause: Dimension Filters

Apply shocks only to rows matching conditions on table dimensions.

```python
from gaspatchio.scenarios import FilteredShock, MultiplicativeShock

# Increase early-duration lapse by 25%
shock = FilteredShock(
    shock=MultiplicativeShock(factor=1.25),
    where={"duration": {"lte": 3}},
    table="lapse",
)

# Mortality shock for elderly lives
shock = FilteredShock(
    shock=MultiplicativeShock(factor=1.15),
    where={"attained_age": {"gte": 65}},
    table="mortality",
)

# Complex filter with multiple conditions (AND logic)
shock = FilteredShock(
    shock=AdditiveShock(delta=0.02),
    where={"sex": "F", "smoker_status": "S"},
    table="mortality",
)
```

**Config syntax:**

```json
{"table": "lapse", "multiply": 1.25, "where": {"duration": {"lte": 3}}}
{"table": "mortality", "multiply": 1.15, "where": {"attained_age": {"gte": 65}}}
```

**Supported filter operators:**

| Operator  | Meaning               | Example                                       |
| --------- | --------------------- | --------------------------------------------- |
| `eq`      | Equals                | `{"sex": {"eq": "F"}}` or just `{"sex": "F"}` |
| `ne`      | Not equals            | `{"product": {"ne": "TERM"}}`                 |
| `gt`      | Greater than          | `{"age": {"gt": 60}}`                         |
| `gte`     | Greater than or equal | `{"duration": {"gte": 5}}`                    |
| `lt`      | Less than             | `{"age": {"lt": 30}}`                         |
| `lte`     | Less than or equal    | `{"duration": {"lte": 3}}`                    |
| `between` | Range (inclusive)     | `{"age": {"between": [30, 50]}}`              |
| `in`      | In list               | `{"product": {"in": ["TERM", "WL"]}}`         |
| `not_in`  | Not in list           | `{"status": {"not_in": ["LAPSED", "DEATH"]}}` |

### WHEN Clause: Time Conditions

Apply shocks only at specific projection times.

```python
from gaspatchio.scenarios import TimeConditionalShock, AdditiveShock

# Mass lapse at t=0 (40% immediate surrender)
shock = TimeConditionalShock(
    shock=AdditiveShock(delta=0.40),
    when={"t": {"eq": 0}},
    table="lapse",
)

# Expense shock for first 5 years only
shock = TimeConditionalShock(
    shock=MultiplicativeShock(factor=1.10),
    when={"t": {"lte": 5}},
    table="expenses",
)
```

**Config syntax:**

```json
{"table": "lapse", "add": 0.40, "when": {"t": {"eq": 0}}}
{"table": "expenses", "multiply": 1.10, "when": {"t": {"lte": 5}}}
```

### Combining WHERE and WHEN

You can use both filters together:

```json
{
    "table": "lapse",
    "multiply": 1.5,
    "where": {"product": "TERM"},
    "when": {"t": {"lte": 5}}
}
```

This reads: "Increase lapse by 50% for TERM products during the first 5 years."

______________________________________________________________________

## Composing Operations

Complex regulatory scenarios often need multiple operations applied in sequence.

### Pipeline Shocks

Chain operations together. Each step's output becomes the next step's input.

```python
from gaspatchio.scenarios import PipelineShock, MultiplicativeShock, ClipShock

# Solvency II lapse up: multiply by 1.5, then cap at 100%
shock = PipelineShock(
    shocks=[
        MultiplicativeShock(factor=1.5),
        ClipShock(max_value=1.0),
    ],
    table="lapse",
)
```

**Config syntax:**

```json
{
    "table": "lapse",
    "pipeline": [
        {"multiply": 1.5},
        {"clip": {"max": 1.0}}
    ]
}
```

**Syntactic sugar** - combine operation + clip in one config:

```json
{"table": "lapse", "multiply": 1.5, "clip": [null, 1.0]}
```

This is equivalent to the pipeline above but more concise.

### Max Shocks

Take the maximum (less severe) of two transformations. Essential for Solvency II lapse down.

```python
from gaspatchio.scenarios import MaxShock, MultiplicativeShock, AdditiveShock

# Solvency II lapse down: max(lapse × 0.5, lapse - 0.2)
# This means: reduce by 50% OR reduce by 20pp, whichever is less severe
shock = MaxShock(
    shock_a=MultiplicativeShock(factor=0.5),
    shock_b=AdditiveShock(delta=-0.2),
    table="lapse",
)
```

**Config syntax:**

```json
{"table": "lapse", "max": [{"multiply": 0.5}, {"add": -0.2}]}
```

### Min Shocks

Take the minimum (more severe) of two transformations.

```python
from gaspatchio.scenarios import MinShock, MultiplicativeShock, OverrideShock

# Cap mortality at 10% even after stress
shock = MinShock(
    shock_a=MultiplicativeShock(factor=1.5),
    shock_b=OverrideShock(value=0.1),
    table="mortality",
)
```

**Config syntax:**

```json
{"table": "mortality", "min": [{"multiply": 1.5}, {"set": 0.1}]}
```

______________________________________________________________________

## Parameter Shocks

Some model inputs are scalar parameters, not assumption tables. Use `ParameterShock` for these.

```python
from gaspatchio.scenarios import ParameterShock

# Add 1% to expense inflation
shock = ParameterShock(param="expense_inflation", operation="add", value=0.01)

# Apply in model code
base_inflation = 0.02
shocked_inflation = shock.apply(base_inflation)  # Returns 0.03
```

**Config syntax:**

```json
{"param": "expense_inflation", "add": 0.01}
{"param": "discount_spread", "multiply": 1.5}
{"param": "commission_rate", "set": 0.05}
```

Parameter shocks are different

`ParameterShock` is NOT a `Shock` subclass. It doesn't generate Polars expressions - instead, it provides an `apply(base_value)` method for your model code to use.

______________________________________________________________________

## Configuration Syntax

The shock system supports LLM-friendly JSON/dict configs that can be generated from natural language.

For the *why* (auditability, reproducibility, and **no regenerated assumption tables**) plus richer actuarial examples, see [Natural Language → Executable Configs](https://gaspatchio.dev/0.9.0/concepts/scenarios/llm-friendly-configs/index.md).

### Parsing Configs

```python
from gaspatchio.scenarios import parse_shock_config, parse_scenario_config

# Parse a single shock
config = {"table": "mortality", "multiply": 1.2}
shock = parse_shock_config(config)
# Returns: MultiplicativeShock(factor=1.2, table="mortality")

# Parse a full scenario configuration
config = [
    {"id": "BASE"},
    {
        "id": "STRESS",
        "shocks": [
            {"table": "mortality", "multiply": 1.2},
            {"table": "lapse", "multiply": 0.8},
        ],
    },
]
scenarios = parse_scenario_config(config)
# Returns: {"BASE": [], "STRESS": [MultiplicativeShock(...), MultiplicativeShock(...)]}
```

### Full Config Schema

```json
{
    "id": "SCENARIO_NAME",
    "shocks": [
        {
            "table": "table_name",
            "column": "optional_column",
            "pipeline": [
                {"multiply": 1.5},
                {"clip": {"max": 1.0}}
            ],
            "where": {"dimension": {"operator": "value"}},
            "when": {"t": {"operator": "value"}}
        }
    ]
}
```

| Field      | Type   | Required | Description                                            |
| ---------- | ------ | -------- | ------------------------------------------------------ |
| `table`    | string | Yes\*    | Target assumption table                                |
| `param`    | string | Yes\*    | Target scalar parameter (\*use one of table/param)     |
| `column`   | string | No       | Specific column within table                           |
| `pipeline` | array  | No       | Sequence of operations                                 |
| `where`    | object | No       | Dimension filter conditions                            |
| `when`     | object | No       | Time filter conditions                                 |
| Operation  | varies | Yes      | One of: `multiply`, `add`, `set`, `clip`, `max`, `min` |

______________________________________________________________________

## Putting It Together: Solvency II SCR

Here's how the building blocks combine into real regulatory scenarios.

### The Solvency II Lapse Shocks

| Shock      | Requirement                     | Config                       |
| ---------- | ------------------------------- | ---------------------------- |
| Lapse Up   | `min(lapse × 1.5, 1.0)`         | Pipeline: multiply then clip |
| Lapse Down | `max(lapse × 0.5, lapse - 0.2)` | Max of two transformations   |
| Mass Lapse | 40% surrender at t=0            | Time-conditional additive    |

```python
import polars as pl
from gaspatchio.assumptions import Table
from gaspatchio.scenarios import parse_scenario_config

# Baseline lapse table the shocks overlay (governed assumptions stay untouched)
lapse_table = Table(
    name="lapse",
    source=pl.DataFrame({
        "duration":   [1, 2, 3, 4, 5],
        "lapse_rate": [0.08, 0.06, 0.05, 0.04, 0.03],
    }),
    dimensions={"duration": "duration"},
    value="lapse_rate",
)

# Define Solvency II lapse scenarios
config = [
    {"id": "BASE"},
    {
        "id": "LAPSE_UP",
        "shocks": [
            {
                "table": "lapse",
                "pipeline": [
                    {"multiply": 1.5},
                    {"clip": {"max": 1.0}}
                ]
            }
        ]
    },
    {
        "id": "LAPSE_DOWN",
        "shocks": [
            {"table": "lapse", "max": [{"multiply": 0.5}, {"add": -0.2}]}
        ]
    },
    {
        "id": "MASS_LAPSE",
        "shocks": [
            {"table": "lapse", "add": 0.40, "when": {"t": {"eq": 0}}}
        ]
    },
]

# Parse into shock objects, then pass to a ScenarioRun for an audit trail.
scenarios = parse_scenario_config(config)

from gaspatchio.scenarios import ScenarioRun, Sum
plan = ScenarioRun(
    shocks=scenarios,
    base_tables={"lapse": lapse_table},
    aggregations=(Sum("bel").alias("bel"),),
)
print(plan.describe())
```

`plan.describe()` emits a human-readable summary of the shocks per scenario; `plan.source_sha()` captures the entire recipe as a content hash for the audit chain.

### Complete Working Example

End-to-end: declare the lapse shocks as a JSON config, parse them into a `ScenarioRun`, run the plan, read per-scenario aggregates.

```python
import polars as pl
from gaspatchio import ActuarialFrame
from gaspatchio.assumptions import Table
from gaspatchio.scenarios import (
    ScenarioRun,
    Sum,
    parse_scenario_config,
)

# Sample lapse table
lapse_table = Table(
    name="lapse",
    source=pl.DataFrame({
        "duration":   [1, 2, 3, 4, 5],
        "lapse_rate": [0.08, 0.06, 0.05, 0.04, 0.03],
    }),
    dimensions={"duration": "duration"},
    value="lapse_rate",
)

# Model points
def policies():
    return ActuarialFrame({
        "policy_id": ["P001", "P002", "P003"],
        "duration":  [1, 3, 5],
    })

# Shock recipes — JSON in, Shock objects out
shock_config = [
    {"id": "BASE"},
    {
        "id": "LAPSE_UP",
        "shocks": [{"table": "lapse", "pipeline": [{"multiply": 1.5}, {"clip": {"max": 1.0}}]}],
    },
    {
        "id": "LAPSE_DOWN",
        "shocks": [{"table": "lapse", "max": [{"multiply": 0.5}, {"add": -0.2}]}],
    },
]
shocks = parse_scenario_config(shock_config)

# Model function — receives shocked tables per scenario. ScenarioRun stacks
# the lapse table with a scenario_id dimension so a single lookup resolves
# the shocked rate for whichever scenario the batch is running.
def model(af, *, tables, drivers=None):
    rate = tables["lapse"].lookup(
        duration=af["duration"],
        scenario_id=af["scenario_id"],
    )
    return af.with_columns(rate.alias("shocked_lapse"))

# Run all three scenarios in one pass
plan = ScenarioRun(
    shocks=shocks,
    base_tables={"lapse": lapse_table},
    aggregations=(Sum("shocked_lapse").alias("total_lapse").over("scenario_id"),),
)
result = plan.run(policies(), model, batch_size=1)
print(result.aggregations["total_lapse"].sort("scenario_id"))
```

______________________________________________________________________

## Framework-Agnostic Patterns

While we used Solvency II as an example, these patterns apply to any regulatory framework:

| Framework       | Common Shocks                                | Gaspatchio Pattern                        |
| --------------- | -------------------------------------------- | ----------------------------------------- |
| **Solvency II** | Lapse up/down, mass lapse, expense inflation | Pipeline + clip, max, time-conditional    |
| **IFRS 17**     | Risk adjustment scenarios                    | Multiplicative on all decrements          |
| **US RBC**      | C1-C4 factors                                | Multiplicative, often filtered by product |
| **ORSA**        | Management actions, stress scenarios         | Combined where + when filters             |
| **ALM**         | Interest rate shocks                         | Additive on curve tables                  |

The building blocks are universal - only the specific factors and combinations change.

______________________________________________________________________

## API Quick Reference

See the [Scenarios API](https://gaspatchio.dev/0.9.0/api/scenarios/index.md) for full function signatures.

| Function/Class            | Purpose                                                    |
| ------------------------- | ---------------------------------------------------------- |
| `ScenarioRun`             | Typed plan: shocks, base tables, aggregations, audit chain |
| `for_each_scenario()`     | Bounded-memory scenario loop with built-in batching        |
| `parse_shock_config()`    | Parse a single shock dict → Shock object                   |
| `parse_scenario_config()` | Parse a scenario list → dict of shocks                     |
| `with_scenarios()`        | Low-level cross-join of ActuarialFrame × scenarios         |

**Shock Classes:**

| Class                  | Operation                                                                  |
| ---------------------- | -------------------------------------------------------------------------- |
| `MultiplicativeShock`  | Scale by factor                                                            |
| `AdditiveShock`        | Add constant                                                               |
| `OverrideShock`        | Replace with value                                                         |
| `ClipShock`            | Cap/floor values                                                           |
| `PipelineShock`        | Chain operations                                                           |
| `FilteredShock`        | WHERE clause                                                               |
| `TimeConditionalShock` | WHEN clause                                                                |
| `MaxShock`             | Maximum of two shocks                                                      |
| `MinShock`             | Minimum of two shocks                                                      |
| `RelativeFloorShock`   | Floor relative to original value (e.g. cannot decrease by more than delta) |
| `ParameterShock`       | Scalar parameter shocks                                                    |

______________________________________________________________________

## See Also

- [What-If Analysis](https://gaspatchio.dev/0.9.0/concepts/scenarios/what-if/index.md) - Simple question → config translation
- [Table Sensitivities](https://gaspatchio.dev/0.9.0/concepts/scenarios/table-sensitivities/index.md) - Python API for applying shocks to tables
- [Scenarios Overview](https://gaspatchio.dev/0.9.0/concepts/scenarios/index.md) - High-level scenario concepts
- [Performance](https://gaspatchio.dev/0.9.0/concepts/scenarios/performance/index.md) - Optimizing large scenario runs

# Streaming and Spill

Some questions only the full grid can answer. A C1/C2 capital charge reads every per-policy cashflow; an audit extract has to reproduce each cell a reviewer might query; a downstream system ingests the projection row by row, not a portfolio total. For these you need the complete per-policy output — every cashflow, every period, every policy — not an aggregate.

The problem is size. A portfolio of millions of policies, each projected over a few hundred periods, is a grid with hundreds of millions of rows. That grid does not fit in memory, and unlike a portfolio total it cannot be folded down as you go — every cell is part of the answer.

`run_to_parquet` is the tool for that case. It projects the portfolio in memory-safe batches and writes each batch straight to disk as a parquet file, so the full output never has to be resident all at once. You get back a manifest describing what was written.

This is the counterpart to [aggregating at scale](https://gaspatchio.dev/0.9.0/concepts/scenarios/aggregating-at-scale/index.md). When the answer you need *is* an aggregate — a sum of reserves, an SCR at the 99.5th percentile — the run folds each batch down to per-period figures and only the running aggregate stays in memory; the per-policy detail is discarded as it goes. `run_to_parquet` is the opposite contract: it keeps every per-policy row, paying for that with disk instead of RAM. Reach for it when you genuinely need the full grid, not a summary of it.

______________________________________________________________________

## Spill a portfolio to disk

A short net-cashflow projection over a four-policy book. The model function builds one per-policy list column; `run_to_parquet` slices the book into batches of two and writes each batch to its own parquet file under `policy_output/`.

```python
from pathlib import Path

import polars as pl
from gaspatchio import ActuarialFrame
from gaspatchio.scenarios import run_to_parquet


def project(af: ActuarialFrame) -> ActuarialFrame:
    """Net cashflow per period: premium less a growing claim cost."""
    months = 3
    af.net_cf = pl.concat_list(
        [pl.col("premium") - pl.col("claim_cost") * (t + 1) for t in range(months)]
    )
    return af


model_points = pl.DataFrame({
    "policy_id":  [1, 2, 3, 4],
    "premium":    [1200.0, 2400.0, 600.0, 1800.0],
    "claim_cost": [50.0, 120.0, 20.0, 90.0],
})

result = run_to_parquet(project, model_points, Path("policy_output"), batch_size=2)

print(result.n_policies)   # 4
print(result.n_batches)    # 2
print(result.output_dir)   # policy_output
```

Four policies, batched two at a time, give two parquet files. Each batch carries the model points it was sliced from plus every column the projection added — here the `net_cf` list column, one entry per projection period.

The files are named `batch_NNNN.parquet`, zero-padded to four digits and numbered from zero, so they sort in projection order on disk:

```python
written = sorted(p.name for p in Path("policy_output").glob("*.parquet"))
print(written)
# ['batch_0000.parquet', 'batch_0001.parquet']
```

Read one back and the per-policy detail is exactly what the projection produced — nothing was aggregated away:

```python
first_batch = pl.read_parquet("policy_output/batch_0000.parquet")
print(first_batch.select(["policy_id", "premium", "net_cf"]))
# policy_id 1: net_cf [1150.0, 1100.0, 1050.0]
# policy_id 2: net_cf [2280.0, 2160.0, 2040.0]
```

Point a lazy scan at the whole directory and you read the full grid back without loading it all at once — the same memory discipline that wrote it:

```python
full_grid = pl.scan_parquet("policy_output/batch_*.parquet")
print(full_grid.select(pl.len()).collect().item())   # 4
```

______________________________________________________________________

## The manifest

`run_to_parquet` returns a `SpillResult` — a small record describing the run, not the data itself. The data is on disk; the manifest tells you where and how the run behaved.

```python
print(result.output_dir)     # PosixPath('policy_output') — where the batches landed
print(result.n_policies)     # 4   — rows projected, total
print(result.n_batches)      # 2   — batch_NNNN.parquet files written
print(round(result.wall_time_s, 3))   # seconds of wall time
print(result.peak_rss_mb)    # peak resident memory above baseline, in MB
```

`n_policies` and `n_batches` reconcile the run: every policy went into exactly one batch, and `n_batches` is the file count you can expect to find under `output_dir`. `peak_rss_mb` is the headroom the run actually used above its starting footprint — the figure that confirms the batching kept the job inside its memory budget rather than the figure you hoped for.

______________________________________________________________________

## Sizing the batches

The `batch_size` argument controls how many policies go into each file.

Pass an integer to fix the batch size yourself, as the example above does with `batch_size=2`. This is the predictable choice when you already know what your hardware will hold, or when you want a fixed number of policies per file for a downstream consumer.

Leave it at the default — `batch_size="auto"` — and the run sizes each batch to the available memory budget. It projects a small leading slice to measure how much memory a single policy's output consumes, checks the target directory has the disk room for the full output, then picks the largest batch that stays within budget. Batching here exists for one reason: memory safety. The full per-policy output cannot fold to a running total, so the run holds one batch at a time and lets the rest live on disk.

```python
# Let the run size each batch to the memory budget.
auto = run_to_parquet(project, model_points, Path("policy_output_auto"))
print(auto.n_policies)   # 4
print(auto.n_batches)    # 1 — this tiny book fits in a single batch
```

A small book like this fits in one batch; a portfolio of millions resolves to many. Either way the contract is the same — every per-policy row is written, and no more than one batch is resident at a time.

______________________________________________________________________

## Where the batches go

`output_dir` is a directory, created if it does not exist, and each batch is written there atomically: the run writes to a temporary file first and renames it into place only once the full batch is on disk, so an interrupted run never leaves a half-written `batch_NNNN.parquet` for a reader to trip over.

One constraint is worth knowing. The point of spilling is to move the full output out of memory and onto a disk, so the target must be real disk. A RAM-backed filesystem defeats the exercise — the batches would count against the same memory the run is trying to protect — and `run_to_parquet` refuses such a target with a clear error rather than spilling into the memory it was asked to spare. Before the first batch is written, the run also checks the filesystem has room for the estimated full output and stops up front if it does not, rather than failing partway through a long projection.

______________________________________________________________________

## When to reach for it

| You need...                                                       | Use                                                                                                   |
| ----------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------- |
| A portfolio total, a percentile, an SCR — an aggregate of the run | [aggregating at scale](https://gaspatchio.dev/0.9.0/concepts/scenarios/aggregating-at-scale/index.md) |
| Every per-policy cashflow, at a scale that won't fit in memory    | `run_to_parquet`                                                                                      |
| Quick exploration on a book that fits in RAM                      | a plain projection on an `ActuarialFrame`                                                             |

The dividing line is the answer, not the portfolio size. If the answer is a summary, fold it down and keep the run in memory. If the answer is the grid itself — a capital extract, an audit trail, a feed to a downstream system — spill it to disk and read it back batch by batch.

# Watching a run converge

A stochastic capital run folds thousands of scenarios over minutes. Until the last batch lands it is a black box: you cannot see whether the 95% CTE has settled, you cannot catch a run that will never converge, and you have nothing to show a reviewer watching over your shoulder. The estimate you actually care about — a tail reserve, a quantile, a mean — is sitting in the accumulator the whole time, but the loop only hands it back at the end.

`for_each_scenario` opens that accumulator up. Pass an `on_batch` callback and, after every batch folds, you receive a `BatchSnapshot` carrying the **running partials** — the same aggregates computed over every scenario seen so far. You read your tail metric off each snapshot and watch it drift toward its final value as the run streams, stop reporting once it has settled, or pipe each snapshot to a live chart.

Both entry points stream this way. `for_each_scenario` — the raw loop — takes `on_batch` directly. A reproducible [`ScenarioRun`](https://gaspatchio.dev/0.9.0/concepts/scenarios/scenariorun/index.md) accepts the same `on_batch` and `progress` on its `.run(...)`: watching a plan converge is a live observation channel and never changes its `source_sha` or audit sidecar. The portfolio folds that return a single summary — [`run_aggregated`](https://gaspatchio.dev/0.9.0/concepts/scenarios/aggregating-at-scale/index.md) and [`run_to_parquet`](https://gaspatchio.dev/0.9.0/concepts/scenarios/streaming-and-spill/index.md) — hand back their results in one piece.

______________________________________________________________________

## A convergence trace

A small book, 500 stochastic scenarios, each applying its own lognormal claim shock. The aggregator folds each scenario's portfolio net cashflow with `.over("scenario_id")`, so the running partial is the loss distribution built so far. The `on_batch` reads the 90% CTE off that distribution after every batch:

```python
import random
import polars as pl
from gaspatchio import ActuarialFrame
from gaspatchio.scenarios import for_each_scenario, Sum

rng = random.Random(7)
book = pl.DataFrame({
    "policy_id": [1, 2, 3, 4, 5],
    "premium":   [1000.0] * 5,
    "claim":     [600.0] * 5,
})

n_scenarios = 500
scenario_ids = list(range(1, n_scenarios + 1))
# each scenario applies its own lognormal claim multiplier — a loss distribution
claim_shock = [rng.lognormvariate(0.0, 0.4) for _ in scenario_ids]


def model(af, *, tables=None, drivers=None):
    """The loop injects this batch's `scenario_id`; key the shock off it."""
    af.net_cf = pl.col("premium") - pl.col("claim") * pl.col("scenario_id").replace_strict(
        old=scenario_ids, new=claim_shock, return_dtype=pl.Float64
    )
    return af


def cte(dist: pl.DataFrame, level: float) -> float:
    """90% CTE of the loss: the mean of the worst (1 - level) of scenarios."""
    loss = -dist["portfolio_cf"]
    threshold = loss.quantile(level)
    return loss.filter(loss >= threshold).mean()


trace = []
def on_batch(snap):
    # snap.outputs["portfolio_cf"] is the running per-scenario distribution so far
    trace.append((snap.scenarios_done, cte(snap.outputs["portfolio_cf"], 0.90)))


result = for_each_scenario(
    ActuarialFrame(book),
    scenarios=scenario_ids,
    model_fn=model,
    aggregations=(Sum("net_cf").alias("portfolio_cf").over("scenario_id"),),
    batch_size=25,                       # explicit and small — see below
    on_batch=on_batch,
)

print("frames:", len(trace))
for done, c in trace[::3]:
    print(f"  after {done:3d} scenarios   running CTE90(loss) = {c:7.1f}")
print("final CTE90:", round(cte(result.aggregations["portfolio_cf"], 0.90), 1))
# frames: 20
#   after  25 scenarios   running CTE90(loss) =   -99.9
#   after 100 scenarios   running CTE90(loss) =   870.4
#   after 175 scenarios   running CTE90(loss) =  1209.1
#   after 250 scenarios   running CTE90(loss) =   986.0
#   after 325 scenarios   running CTE90(loss) =   917.3
#   after 400 scenarios   running CTE90(loss) =  1104.1
#   after 475 scenarios   running CTE90(loss) =  1175.1
# final CTE90: 1178.5
```

The shape is the point. At 25 scenarios the 90% CTE is meaningless — there are barely two scenarios in the tail. By a few hundred it has climbed into a tight band and, by the last batch, settled on `1178.5`. You are reading the *exact* figure over the scenarios folded so far at each step, not an approximation: frame *K* is the true CTE of the first *K* batches.

______________________________________________________________________

## What each snapshot carries

`on_batch` receives a frozen `BatchSnapshot` at the end of every batch:

| Field             | Meaning                                                                                                                                                              |
| ----------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `batch_idx`       | 0-based batch index (the loop's enumerate index).                                                                                                                    |
| `scenarios_done`  | Cumulative real scenarios folded so far, inclusive of this batch. Probe scenarios are never counted.                                                                 |
| `total_scenarios` | Total scenarios in the run.                                                                                                                                          |
| `outputs`         | `{alias: running partial}` — the aggregate over every scenario seen so far. A scalar for a plain aggregator; a `pl.DataFrame` for one partitioned with `.over(...)`. |
| `peak_rss_mb`     | Peak resident memory at this point in the run, or `None`.                                                                                                            |
| `elapsed_s`       | Wall seconds since the run started, measured at the end of this batch.                                                                                               |
| `fraction_done`   | `scenarios_done / total_scenarios`, in `[0, 1]` (`0.0` for an empty run).                                                                                            |
| `eta_s`           | Rough seconds remaining, by linear extrapolation from `elapsed_s` and `fraction_done`. `None` before any scenario folds; `0.0` once the run completes.               |
| `throughput`      | Real scenarios folded per second so far; `None` before any time has elapsed.                                                                                         |

`elapsed_s` is a stored field; `fraction_done`, `eta_s` and `throughput` are derived from it and the two counters — enough to drive a progress bar without touching the partials. Treat `eta_s` as a guide, not a clock: under `batch_size="auto"` the batches vary in size (and the search probes deliberately differ), so the estimate is approximate.

The values in `outputs` are materialised, not live accumulator state — a later batch never reaches back and mutates a snapshot you have already stored, so it is safe to keep every frame.

______________________________________________________________________

## Sizing the batches so you can see it

To watch a metric converge, set an **explicit small `batch_size`**. The default `batch_size="auto"` sizes each batch to a memory budget, which on a small book folds the whole run in a handful of batches — a handful of snapshots, no visible convergence. `batch_size=25` over 500 scenarios gives 20 frames; pick a size that trades frame count against the per-batch overhead you can afford.

Two more properties make the hook safe to wire into a long run:

- **It cannot break the run.** An exception raised inside your `on_batch` is swallowed — the run completes and returns its `ScenarioResult` regardless. A broken dashboard never costs you a capital run.
- **`progress=True` gives you a default.** Passing `progress=True` without your own `on_batch` installs a built-in handler that logs each batch as `scenarios {done}/{total} ({percent}) · ETA {time}` — the ETA drops off the line once the run completes. If you pass both, your `on_batch` wins.

______________________________________________________________________

## Streaming to a live view

Because every snapshot is the exact figure over the scenarios folded so far, the same hook drives a live dashboard. The pattern is deliberately plain — no framework, no socket:

1. In `on_batch`, append one JSON line per batch (your metric, plus a histogram of the running distribution) to a file, flushing after each write so a reader never sees a half-written record.
1. Serve the file and a small page next to it. The page polls the file every few hundred milliseconds and redraws a chart from the lines it has so far.

```python
# docs-skip
import json

def on_batch(snap):
    dist = snap.outputs["portfolio_cf"]
    loss = -dist["portfolio_cf"]
    frame = {
        "scenarios_done": snap.scenarios_done,
        "total": snap.total_scenarios,
        "cte90": cte(dist, 0.90),
        "hist": loss.hist(bin_count=40)["count"].to_list(),
    }
    stream.write(json.dumps(frame) + "\n")
    stream.flush()   # so a polling reader never reads a torn line
```

A browser polling that file fills in the convergence trace and the loss histogram as the run streams — the left panel settling onto the final reserve, the right panel filling out the tail:

Two details keep the picture honest: **freeze the histogram range** on the first full batch (per-frame auto-ranging rescales the bars every tick), and keep the writer **append-only with a flush per line** so the polling reader always parses clean JSON. The Gaspatchio repository ships a worked example of exactly this — a convergence demo plus a dependency-free viewer — under `evals/benchmarks/`.

______________________________________________________________________

## When to reach for `on_batch`

| You want...                                                 | Use                                                                                                           |
| ----------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------- |
| Watch a tail metric settle and stop once it has             | `for_each_scenario(..., on_batch=...)`, read the metric off each snapshot                                     |
| A live convergence chart for a reviewer                     | `on_batch` appending JSONL frames, polled by a page                                                           |
| A progress log and nothing more                             | `for_each_scenario(..., progress=True)`                                                                       |
| A progress log on a reproducible run                        | [`ScenarioRun.run(..., progress=True)`](https://gaspatchio.dev/0.9.0/concepts/scenarios/scenariorun/index.md) |
| Watch a reproducible, hashable run converge                 | [`ScenarioRun.run(..., on_batch=...)`](https://gaspatchio.dev/0.9.0/concepts/scenarios/scenariorun/index.md)  |
| The final portfolio summary, memory-safe, no per-batch hook | [`run_aggregated`](https://gaspatchio.dev/0.9.0/concepts/scenarios/aggregating-at-scale/index.md)             |

# Table Sensitivities

This page is the low-level Python reference for the shock primitives applied to a single `Table`. For multi-scenario stress runs — where the same shock list applies to many scenarios with a single audit chain — declare the shocks inside a [`ScenarioRun`](https://gaspatchio.dev/0.9.0/concepts/scenarios/scenariorun/index.md) instead; it stacks tables across scenarios for you.

Reach for `Table.with_shock()` when you're applying a single, ad-hoc shock to a single table — typically for interactive exploration or for one-off comparisons where you don't need the full scenario-run plumbing.

## Single-Table Shock

```python
import polars as pl
from gaspatchio import ActuarialFrame
from gaspatchio.assumptions import Table
from gaspatchio.scenarios.shocks import MultiplicativeShock

# Load your existing base table
mortality = Table(
    name="mortality",
    source="assumptions/mortality.parquet",
    dimensions={"age": "age", "duration": "duration"},
    value="qx"
)

# Create a shocked version (original unchanged)
stressed_mortality = mortality.with_shock(MultiplicativeShock(factor=1.2))

# Use in model
af = ActuarialFrame(pl.read_parquet("model_points.parquet"))
af.mort_rate = stressed_mortality.lookup(age=af.age, duration=af.duration)
```

`with_shock()` returns a **new** `Table` — the original is never modified. For multi-scenario runs prefer `ScenarioRun(shocks={…}, base_tables={…}, …)` — see [Scenarios Overview](https://gaspatchio.dev/0.9.0/concepts/scenarios/index.md).

______________________________________________________________________

## Shock Classes

Gaspatchio provides three shock types:

### MultiplicativeShock

Scales values by a factor. Use for percentage changes.

```python
from gaspatchio.scenarios.shocks import MultiplicativeShock

# 20% increase (multiply by 1.2)
shock = MultiplicativeShock(factor=1.2, table="mortality")

# 10% decrease (multiply by 0.9)
shock = MultiplicativeShock(factor=0.9, table="lapse")

# Target specific column within table
shock = MultiplicativeShock(factor=1.2, table="mortality", column="qx")
```

| Parameter | Type  | Description                                    |
| --------- | ----- | ---------------------------------------------- |
| `factor`  | float | Multiplicative factor (1.2 = +20%, 0.8 = -20%) |
| `table`   | str   | Target table name                              |
| `column`  | str   | Optional: specific column to shock             |

### AdditiveShock

Adds a constant delta. Use for basis point shifts.

```python
from gaspatchio.scenarios.shocks import AdditiveShock

# Add 50 basis points
shock = AdditiveShock(delta=0.005, table="discount_rates")

# Subtract 100 basis points
shock = AdditiveShock(delta=-0.01, table="discount_rates")
```

| Parameter | Type  | Description                         |
| --------- | ----- | ----------------------------------- |
| `delta`   | float | Additive constant (+0.01 = +100bps) |
| `table`   | str   | Target table name                   |
| `column`  | str   | Optional: specific column to shock  |

### OverrideShock

Replaces all values with a constant. Use for extreme scenarios.

```python
from gaspatchio.scenarios.shocks import OverrideShock

# Zero out all lapses
shock = OverrideShock(value=0.0, table="lapse")

# Flat 5% discount rate
shock = OverrideShock(value=0.05, table="discount_rates")
```

| Parameter | Type | Description                        |
| --------- | ---- | ---------------------------------- |
| `value`   | Any  | Constant replacement value         |
| `table`   | str  | Target table name                  |
| `column`  | str  | Optional: specific column to shock |

______________________________________________________________________

## Applying Shocks to Tables

### Single Shock

```python
# `mortality` is the base Table loaded above; `shock` is any Shock instance.
base_table = mortality
shock = MultiplicativeShock(factor=1.2)

# Apply one shock
stressed = base_table.with_shock(shock)

# With custom name
stressed = base_table.with_shock(shock, name="mortality_stressed")
```

### Chaining Multiple Shocks

Shocks apply sequentially:

```python
from gaspatchio.scenarios.shocks import MultiplicativeShock, AdditiveShock

# First multiply by 2, then add 0.001
double_shocked = mortality.with_shock(
    MultiplicativeShock(factor=2.0)
).with_shock(
    AdditiveShock(delta=0.001)
)
# Original value 0.001 becomes: 0.001 * 2 + 0.001 = 0.003
```

### Batch Creation with from_shocks()

Create multiple shocked tables at once:

```python
from gaspatchio.assumptions import Table
from gaspatchio.scenarios.shocks import MultiplicativeShock

# `base_mortality` is your governed base table (the `mortality` Table above).
base_mortality = mortality

shock_specs = {
    "BASE": [],  # Empty = no shocks
    "UP_20": [MultiplicativeShock(factor=1.2)],
    "DOWN_20": [MultiplicativeShock(factor=0.8)],
}

# Returns dict of Tables
tables = Table.from_shocks(base_mortality, shock_specs, value_column="qx")
# tables["BASE"], tables["UP_20"], tables["DOWN_20"]
```

______________________________________________________________________

## Config Parsing Functions

### parse_scenario_config()

Converts JSON config to shock objects:

```python
from gaspatchio.scenarios import parse_scenario_config

config = [
    {"id": "BASE"},
    {"id": "MORT_UP_20", "shocks": [{"table": "mortality", "multiply": 1.2}]},
    {"id": "RATES_DOWN", "shocks": [{"table": "discount_rates", "add": -0.005}]},
]

scenarios = parse_scenario_config(config)
# Returns: dict[str, list[Shock]]
# {
#     "BASE": [],
#     "MORT_UP_20": [MultiplicativeShock(factor=1.2, table="mortality")],
#     "RATES_DOWN": [AdditiveShock(delta=-0.005, table="discount_rates")],
# }
```

### Plan-level audit trail

For an audit-ready description of a scenario set, build a `ScenarioRun` and call `.describe()`:

```python
from gaspatchio.scenarios import ScenarioRun, Sum

# The shocks reference the `mortality` and `discount_rates` tables, so the plan
# needs the matching base tables to hash and describe them.
base_tables = {
    "mortality": Table(
        name="mortality",
        source="assumptions/mortality.parquet",
        dimensions={"age": "age", "duration": "duration"},
        value="qx",
    ),
    "discount_rates": Table(
        name="discount_rates",
        source="assumptions/discount_rates.parquet",
        dimensions={"year": "year"},
        value="rate",
    ),
}

plan = ScenarioRun(
    shocks=scenarios,
    base_tables=base_tables,
    aggregations=(Sum("bel").alias("bel"),),
)
print(plan.describe())
```

The same plan emits a content hash (`plan.source_sha()`) and, when run with `audit=True`, a JSON sidecar suitable for handoff to model risk.

______________________________________________________________________

## Sensitivity Analysis Helper

For parameter sweeps, build a list of `ScenarioRun` configurations directly — typically a base plan plus shocks generated in a loop. Example:

```python
from gaspatchio.scenarios import MultiplicativeShock, AdditiveShock

# Mortality sensitivity sweep
mortality_sweep = {
    "BASE": [],
    **{
        f"mortality_{v}": [MultiplicativeShock(factor=v, table="mortality")]
        for v in (0.8, 0.9, 1.0, 1.1, 1.2)
    },
}

# Interest rate parallel shifts
rate_sweep = {
    "BASE": [],
    **{
        f"rates_{v:+}": [AdditiveShock(delta=v, table="discount_rates")]
        for v in (-0.01, -0.005, 0.0, 0.005, 0.01)
    },
}
```

Then build one `ScenarioRun` per sweep, or pass the dict directly to `for_each_scenario` if you don't need governance.

______________________________________________________________________

## Complete Workflow Example

### Step 1: Load Base Tables

```python
import polars as pl
from gaspatchio.assumptions import Table

base_tables = {
    "mortality": Table(
        name="mortality",
        source=pl.read_parquet("assumptions/mortality.parquet"),
        dimensions={"age": "age", "duration": "duration"},
        value="qx"
    ),
    "lapse": Table(
        name="lapse",
        source=pl.read_parquet("assumptions/lapse.parquet"),
        dimensions={"duration": "duration"},
        value="rate"
    ),
    "discount_rates": Table(
        name="discount_rates",
        source=pl.read_parquet("assumptions/discount_rates.parquet"),
        dimensions={"year": "year"},
        value="rate"
    ),
}
```

### Step 2: Parse Scenario Config

```python
from gaspatchio.scenarios import parse_scenario_config

config = [
    {"id": "BASE"},
    {"id": "ADVERSE", "shocks": [
        {"table": "mortality", "multiply": 1.2},
        {"table": "lapse", "multiply": 0.8},
        {"table": "discount_rates", "add": -0.01}
    ]},
]

scenarios = parse_scenario_config(config)
```

### Step 3: Run Model for Each Scenario

```python
from gaspatchio import ActuarialFrame

def run_scenario(scenario_id, shocks, base_tables, model_points):
    """Run model with shocked tables for one scenario."""

    # Create shocked tables for this scenario
    tables = base_tables.copy()
    for shock in shocks:
        if shock.table in tables:
            tables[shock.table] = tables[shock.table].with_shock(shock)

    # Run model with shocked tables
    af = ActuarialFrame(model_points)
    af.scenario_id = scenario_id

    # Lookups use shocked tables
    af.mort_rate = tables["mortality"].lookup(age=af.age, duration=af.duration)
    af.lapse_rate = tables["lapse"].lookup(duration=af.duration)
    af.disc_rate = tables["discount_rates"].lookup(year=af.year)

    # ... rest of model calculations

    return af.collect()

# Run all scenarios
model_points = pl.read_parquet("model_points.parquet")
results = []

for scenario_id, shocks in scenarios.items():
    result = run_scenario(scenario_id, shocks, base_tables, model_points)
    results.append(result)

# Combine results
all_results = pl.concat(results)
```

______________________________________________________________________

## Batch Processing

Memory-efficient processing of many scenarios is handled by the [`for_each_scenario`](https://gaspatchio.dev/0.9.0/concepts/scenarios/index.md) loop, which batches scenarios automatically and merges aggregator state across batches.

```python
from gaspatchio import ActuarialFrame
from gaspatchio.scenarios import Sum, for_each_scenario


def run_model(af, *, tables=None, drivers=None):
    """Per-scenario projection — here, net cash flow is simply the premium."""
    return af.with_columns(af["premium"].alias("pv_net_cf"))


scenario_ids = list(range(1, 1_001))  # one thousand scenarios

result = for_each_scenario(
    ActuarialFrame(model_points),
    scenarios=scenario_ids,
    model_fn=run_model,
    aggregations=(Sum("pv_net_cf").alias("total_pv"),),
    batch_size="auto",  # the loop sizes batches to fit memory
)
print(result.aggregations["total_pv"])
```

The aggregator output is bit-equivalent across batch sizes — pick `batch_size` to fit your memory budget without changing the answer. See [Performance at Scale](https://gaspatchio.dev/0.9.0/concepts/scenarios/performance/index.md) for the memory tradeoff details.

______________________________________________________________________

## Best Practices

### 1. Include BASE

A BASE scenario gives the unshocked baseline that other scenarios compare against:

```python
# docs-skip
config = [
    {"id": "BASE"},  # Listed first by convention
    {"id": "STRESS_1", "shocks": [...]},
]
```

### 2. Use Consistent Table Names

Standardize table names across your models:

| Table Name       | Description             |
| ---------------- | ----------------------- |
| `mortality`      | Death rates             |
| `lapse`          | Withdrawal rates        |
| `discount_rates` | Interest/discount rates |
| `expense`        | Per-policy expenses     |
| `inflation`      | Expense inflation       |

### 3. Validate Before Running

Parse the config and inspect the plan before execution:

```python
from gaspatchio.scenarios import ScenarioRun, Sum

scenarios = parse_scenario_config(config)
plan = ScenarioRun(
    shocks=scenarios,
    base_tables=base_tables,
    aggregations=(Sum("bel").alias("bel"),),
)
print(plan.describe())  # Review the shocks and aggregations
print(plan.source_sha())  # Content hash for the audit chain
```

### 4. Store Configs for Reproducibility

Save the plan as YAML alongside results:

```python
from pathlib import Path


def model_fn(af, *, tables=None, drivers=None):
    """Expected claims = premium × shocked mortality for this scenario."""
    qx = tables["mortality"].lookup(
        scenario_id=af["scenario_id"],
        age=af["age"],
        duration=af["duration"],
    )
    return af.with_columns((af["premium"] * qx).alias("bel"))


plan.to_yaml(Path("scenario_plan.yaml"))

# When you run with audit=True, a JSON sidecar lands beside the outputs
# carrying source_sha, library versions, and every aggregator output.
result = plan.run(af, model_fn, batch_size=1, audit=True)
print(result.audit_path)
```

______________________________________________________________________

## See Also

- [What-If Analysis](https://gaspatchio.dev/0.9.0/concepts/scenarios/what-if/index.md) - Declarative config format and examples
- [Scenarios Overview](https://gaspatchio.dev/0.9.0/concepts/scenarios/index.md) - High-level scenario concepts
- [Performance](https://gaspatchio.dev/0.9.0/concepts/scenarios/performance/index.md) - Optimizing large scenario runs

# What-If Analysis

Gaspatchio supports a declarative config format that lets you define scenario shocks without writing Python code. This is designed for:

- **Actuaries**: Ask natural language questions about assumption changes
- **LLMs**: Generate executable scenario configs from user questions
- **Audit**: Clear, JSON-serializable shock specifications

For deeper context on *why* configs matter (and why you **don’t need to regenerate assumption tables**) plus more realistic actuarial prompts, see [Natural Language → Executable Configs](https://gaspatchio.dev/0.9.0/concepts/scenarios/llm-friendly-configs/index.md).

## The Concept

Instead of creating separate assumption files for each scenario or writing Python shock classes, you specify simple JSON configs:

```json
{"table": "mortality", "multiply": 1.2}
```

The framework parses this into shock objects and applies them to your existing tables automatically. Your base assumptions stay untouched - shocks create modified copies in memory when the model runs.

______________________________________________________________________

## Asking What-If Questions

Here are example questions and the configs they translate to:

### "What if mortality increases by 20%?"

```json
[
    {"id": "BASE"},
    {"id": "MORT_UP_20", "shocks": [{"table": "mortality", "multiply": 1.2}]}
]
```

### "What happens if lapse rates drop by half?"

```json
[
    {"id": "BASE"},
    {"id": "LAPSE_DOWN_50", "shocks": [{"table": "lapse", "multiply": 0.5}]}
]
```

### "Show me the impact of a flat 5% discount rate"

```json
[
    {"id": "BASE"},
    {"id": "DISC_FLAT_5PCT", "shocks": [{"table": "disc_rates", "set": 0.05}]}
]
```

### "What if interest rates increase by 100 basis points?"

```json
[
    {"id": "BASE"},
    {"id": "RATES_UP_100BPS", "shocks": [{"table": "disc_rates", "add": 0.01}]}
]
```

### "What's the worst case if mortality is 20% higher AND lapses drop 10%?"

```json
[
    {"id": "BASE"},
    {"id": "ADVERSE", "shocks": [
        {"table": "mortality", "multiply": 1.2},
        {"table": "lapse", "multiply": 0.9}
    ]}
]
```

______________________________________________________________________

## Config Format Reference

### Scenario Config Structure

A scenario config is a **list** where each element defines a scenario:

```json
[
    {"id": "SCENARIO_NAME"},
    {"id": "SCENARIO_NAME", "shocks": [shock, ...]}
]
```

| Field    | Type   | Required | Description                    |
| -------- | ------ | -------- | ------------------------------ |
| `id`     | string | Yes      | Unique scenario identifier     |
| `shocks` | list   | No       | List of shock configs to apply |

### Shock Config Structure

Each shock specifies **which table** to shock and **how** to shock it:

```json
{
    "table": "table_name",
    "column": "column_name",
    "multiply": 1.2
}
```

| Field     | Type   | Required | Description                                |
| --------- | ------ | -------- | ------------------------------------------ |
| `table`   | string | Yes      | Target assumption table name               |
| `column`  | string | No       | Specific column (defaults to value column) |
| Operation | float  | Yes      | One of: `multiply`, `add`, or `set`        |

### Shock Operations

| Operation  | Effect             | Example                                   | Result       |
| ---------- | ------------------ | ----------------------------------------- | ------------ |
| `multiply` | Scale by factor    | `{"table": "mortality", "multiply": 1.2}` | value * 1.2  |
| `add`      | Add constant       | `{"table": "rates", "add": 0.01}`         | value + 0.01 |
| `set`      | Replace with value | `{"table": "lapse", "set": 0.0}`          | value = 0.0  |

**Rules:**

- Exactly **one** operation per shock (`multiply`, `add`, or `set`)
- Multiple shocks can target the same table (applied sequentially)
- Different shocks can target different tables in the same scenario

______________________________________________________________________

## Realistic Scenario Examples

### Interest Rate Sensitivity

**Question**: "Show me PV impact of rates moving +/-50bps and +/-100bps"

```json
[
    {"id": "BASE"},
    {"id": "RATES_DOWN_100BPS", "shocks": [{"table": "disc_rates", "add": -0.01}]},
    {"id": "RATES_DOWN_50BPS", "shocks": [{"table": "disc_rates", "add": -0.005}]},
    {"id": "RATES_UP_50BPS", "shocks": [{"table": "disc_rates", "add": 0.005}]},
    {"id": "RATES_UP_100BPS", "shocks": [{"table": "disc_rates", "add": 0.01}]}
]
```

### Mortality Sensitivity

**Question**: "What's the impact of mortality being 10%, 20%, or 30% higher than expected?"

```json
[
    {"id": "BASE"},
    {"id": "MORT_UP_10", "shocks": [{"table": "mortality", "multiply": 1.1}]},
    {"id": "MORT_UP_20", "shocks": [{"table": "mortality", "multiply": 1.2}]},
    {"id": "MORT_UP_30", "shocks": [{"table": "mortality", "multiply": 1.3}]}
]
```

### Combined Stress Scenarios

**Question**: "Show me best case, base case, and worst case scenarios"

```json
[
    {"id": "BEST_CASE", "shocks": [
        {"table": "mortality", "multiply": 0.9},
        {"table": "lapse", "multiply": 1.1},
        {"table": "disc_rates", "add": 0.005}
    ]},
    {"id": "BASE"},
    {"id": "WORST_CASE", "shocks": [
        {"table": "mortality", "multiply": 1.2},
        {"table": "lapse", "multiply": 0.8},
        {"table": "disc_rates", "add": -0.01}
    ]}
]
```

### Regulatory Stress Test

**Question**: "Run the standard regulatory stress scenarios"

```json
[
    {"id": "BASE"},
    {"id": "EQUITY_DOWN_40", "shocks": [{"table": "equity_returns", "multiply": 0.6}]},
    {"id": "RATES_DOWN_200BPS", "shocks": [{"table": "disc_rates", "add": -0.02}]},
    {"id": "MORT_PANDEMIC", "shocks": [{"table": "mortality", "multiply": 1.5}]},
    {"id": "COMBINED_STRESS", "shocks": [
        {"table": "equity_returns", "multiply": 0.8},
        {"table": "disc_rates", "add": -0.01},
        {"table": "mortality", "multiply": 1.2},
        {"table": "lapse", "multiply": 1.3}
    ]}
]
```

### Flat Assumption Override

**Question**: "What if we assume zero lapses and mortality?"

```json
[
    {"id": "BASE"},
    {"id": "NO_DECREMENTS", "shocks": [
        {"table": "mortality", "set": 0.0},
        {"table": "lapse", "set": 0.0}
    ]}
]
```

______________________________________________________________________

## Common Table Names

When generating configs, use these standard table names:

| Table Name       | Description                 | Typical Shocks                     |
| ---------------- | --------------------------- | ---------------------------------- |
| `mortality`      | Death rates by age/duration | `multiply` (+/-10-50%)             |
| `lapse`          | Withdrawal/surrender rates  | `multiply` (+/-20-50%)             |
| `disc_rates`     | Discount/interest rates     | `add` (+/-50-200bps)               |
| `expense`        | Per-policy expenses         | `multiply` (+/-10-30%)             |
| `inflation`      | Expense inflation           | `add` (+/-1-3%)                    |
| `equity_returns` | Fund returns                | `multiply` (stress), `add` (drift) |
| `premium_rates`  | Premium loading             | `multiply` (rare)                  |

______________________________________________________________________

## Shock Magnitude Guidelines

Typical ranges for sensitivity analysis:

| Assumption     | Typical Range         | Notes                        |
| -------------- | --------------------- | ---------------------------- |
| Mortality      | +/-10% to +/-50%      | Higher for pandemic stress   |
| Lapse          | +/-20% to +/-50%      | Direction depends on product |
| Interest rates | +/-50bps to +/-200bps | Use `add` not `multiply`     |
| Expenses       | +/-10% to +/-30%      | Often paired with inflation  |
| Equity         | -20% to -40%          | Stress scenarios             |

______________________________________________________________________

## Naming Conventions

Use clear, descriptive scenario IDs:

| Pattern                        | Example                | Use Case                |
| ------------------------------ | ---------------------- | ----------------------- |
| `{TABLE}_{DIRECTION}_{AMOUNT}` | `MORT_UP_20`           | Single assumption shock |
| `{DIRECTION}_{AMOUNT}BPS`      | `RATES_DOWN_100BPS`    | Interest rate moves     |
| Descriptive                    | `ADVERSE`, `BEST_CASE` | Combined scenarios      |
| Regulatory                     | `SFCR_STRESS_1`        | Standard tests          |

______________________________________________________________________

## Best Practices

### Include BASE

A BASE scenario gives the unshocked baseline that other scenarios compare against:

```json
[
    {"id": "BASE"},
    {"id": "STRESS_1", "shocks": [...]},
    {"id": "STRESS_2", "shocks": [...]}
]
```

### Validation Errors

The parser validates configs and provides clear error messages:

| Error                                             | Cause                          |
| ------------------------------------------------- | ------------------------------ |
| "Shock config must include 'table' key"           | Missing `table` field          |
| "Shock config must include exactly one operation" | Missing or multiple operations |
| "Duplicate scenario ID"                           | Same `id` used twice           |

______________________________________________________________________

## LLM Integration

The config format is designed for LLM generation. An LLM can translate natural language to executable configs:

| User Question                      | Generated Config                          |
| ---------------------------------- | ----------------------------------------- |
| "What if mortality is 20% higher?" | `{"table": "mortality", "multiply": 1.2}` |
| "Add 50bps to discount rates"      | `{"table": "disc_rates", "add": 0.005}`   |
| "Set lapse rates to zero"          | `{"table": "lapse", "set": 0.0}`          |
| "What's the worst case?"           | Multiple shocks combined                  |

This enables actuaries to ask questions in plain English and get executable scenario analysis without writing code.

______________________________________________________________________

## See Also

- [Shock Operations](https://gaspatchio.dev/0.9.0/concepts/scenarios/shocks/index.md) - Advanced shocks: clip, pipeline, filters, regulatory scenarios
- [Table Sensitivities](https://gaspatchio.dev/0.9.0/concepts/scenarios/table-sensitivities/index.md) - Python API for applying shocks
- [Scenarios Overview](https://gaspatchio.dev/0.9.0/concepts/scenarios/index.md) - High-level scenario concepts
- [Performance](https://gaspatchio.dev/0.9.0/concepts/scenarios/performance/index.md) - Optimizing large scenario runs
# Command line

# Command line (gspio)

You have written a model. Now you want to run it over a portfolio, trace a single life that disagrees with your spreadsheet, check a data file before you trust it, or look up the accessor or the regulation you need while you work — without writing a runner script each time.

`gspio` is the command installed with gaspatchio. It does all of the above from the shell:

```bash
gspio --help
```

```text
 Model Execution
   run-model           Execute an actuarial model from a file
   run-single-policy   Execute an actuarial model for a single policy
   calc-graph          Generate a calculation graph from a model run
 Data Inspection
   describe            Describe the structure of a data file
 Knowledge Discovery
   docs                Search Gaspatchio framework documentation
   knowledge           Search the actuarial knowledge base
 Tutorial
   tutorial            List, initialize, and verify gaspatchio tutorials
```

______________________________________________________________________

## Two inputs every run takes

Every model run takes two files:

- a **model file** — a `.py` with a `main(af)` function that takes an `ActuarialFrame` and returns it with your results assigned;
- a **model-points file** — a `.parquet`, one row per policy.

The fastest way to get both is to copy a worked model out of the box. `gspio tutorial init level-1` writes the Hello World term-life model used throughout this page; pair it with a small portfolio of three policies:

```python
import polars as pl

pl.DataFrame(
    {
        "policy_id":      ["POL001", "POL002", "POL003"],
        "age":            [30, 45, 60],
        "sex":            ["M", "F", "M"],
        "sum_assured":    [500_000, 250_000, 100_000],
        "annual_premium": [450, 1_200, 2_800],
        "mortality_rate": [0.001, 0.004, 0.015],  # annual qx
        "expense_rate":   [0.10, 0.10, 0.10],     # share of premium
    }
).write_parquet("model_points.parquet")
```

The `model.py` you initialised computes expected claims, expenses, net premium, profit, and a loss ratio for each policy. Everything below runs against these two files. For a full production model that ships its own assumptions and model points, `gspio tutorial init level-4` writes a lifelib book reconciled to the reference cashflows.

______________________________________________________________________

## Run a model over the portfolio — `run-model`

Project every policy and read the portfolio result in one command:

```bash
gspio run-model model.py model_points.parquet
```

```text
Result (Columns filtered by -f/-l):
shape: (3, 13)
┌───────────┬─────┬─────┬─────────────┬────────────────┬────────────────┬──────────────┬─────────────────┬──────────┬─────────────┬────────┬────────────┬───────────────┐
│ policy_id ┆ age ┆ sex ┆ sum_assured ┆ annual_premium ┆ mortality_rate ┆ expense_rate ┆ expected_claims ┆ expenses ┆ net_premium ┆ profit ┆ loss_ratio ┆ is_profitable │
│ str       ┆ i64 ┆ str ┆ i64         ┆ i64            ┆ f64            ┆ f64          ┆ f64             ┆ f64      ┆ f64         ┆ f64    ┆ f64        ┆ str           │
╞═══════════╪═════╪═════╪═════════════╪════════════════╪════════════════╪══════════════╪═════════════════╪══════════╪═════════════╪════════╪════════════╪═══════════════╡
│ POL001    ┆ 30  ┆ M   ┆ 500000      ┆ 450            ┆ 0.001          ┆ 0.1          ┆ 500.0           ┆ 45.0     ┆ 405.0       ┆ -95.0  ┆ 1.1111     ┆ No            │
│ POL002    ┆ 45  ┆ F   ┆ 250000      ┆ 1200           ┆ 0.004          ┆ 0.1          ┆ 1000.0          ┆ 120.0    ┆ 1080.0      ┆ 80.0   ┆ 0.8333     ┆ Yes           │
│ POL003    ┆ 60  ┆ M   ┆ 100000      ┆ 2800           ┆ 0.015          ┆ 0.1          ┆ 1500.0          ┆ 280.0    ┆ 2520.0      ┆ 1020.0 ┆ 0.5357     ┆ Yes           │
└───────────┴─────┴─────┴─────────────┴────────────────┴────────────────┴──────────────┴─────────────────┴──────────┴─────────────┴────────┴────────────┴───────────────┘
```

The result is the model points plus every column your model computed. A wide result is shown as a window — the first and last columns, with the middle elided — so the table fits your terminal.

| Option                | Effect                                                                                                                        |
| --------------------- | ----------------------------------------------------------------------------------------------------------------------------- |
| `--mode`, `-m`        | `debug` (default) records each step so you can inspect the calculation; `optimize` skips the bookkeeping for the fastest run. |
| `--first-n`, `-f`     | Number of leading columns to show (default 5).                                                                                |
| `--last-n`, `-l`      | Number of trailing columns to show (default 10).                                                                              |
| `--start-at`, `-s`    | Column index to start the window at (default 0).                                                                              |
| `--rows`, `-r`        | Number of rows to show (default 15).                                                                                          |
| `--output-file`, `-o` | Write the full result to a parquet file instead of printing.                                                                  |

To keep the full result rather than a printed window, write it out:

```bash
gspio run-model model.py model_points.parquet --mode optimize --output-file results.parquet
```

______________________________________________________________________

## Trace a single policy — `run-single-policy`

When one life disagrees with your existing model, run that policy on its own and read its result back column by column:

```bash
gspio run-single-policy model.py model_points.parquet POL001 --policy-id-column policy_id
```

```text
Transposed Result (Columns filtered by -f/-l):
shape: (1, 13)
┌───────────┬─────┬─────┬─────────────┬───┬─────────────┬────────┬────────────┬───────────────┐
│ policy_id ┆ age ┆ sex ┆ sum_assured ┆ … ┆ net_premium ┆ profit ┆ loss_ratio ┆ is_profitable │
│ str       ┆ i64 ┆ str ┆ i64         ┆   ┆ f64         ┆ f64    ┆ f64        ┆ str           │
╞═══════════╪═════╪═════╪═════════════╪═══╪═════════════╪════════╪════════════╪═══════════════╡
│ POL001    ┆ 30  ┆ M   ┆ 500000      ┆ … ┆ 405.0       ┆ -95.0  ┆ 1.1111     ┆ No            │
└───────────┴─────┴─────┴─────────────┴───┴─────────────┴────────┴────────────┴───────────────┘
```

The policy id is the third argument. `--policy-id-column` names the column that holds it — it defaults to `Policy number`, so set it to match your data (`policy_id` here). For a projection model with a time dimension, this returns one row per period: the full trace for that life, which is what you reconcile against your spreadsheet step by step. The same `--mode` and display options as `run-model` apply.

______________________________________________________________________

## Export the calculation graph — `calc-graph`

To audit what feeds what — which inputs and which intermediate columns each result depends on — export the calculation graph:

```bash
gspio calc-graph model.py model_points.parquet --policy-id-column policy_id -o graph.json
```

```text
✓ Calculation graph saved to: graph.json
  Nodes: 13 (7 inputs, 6 computed)
  Edges: 11
```

`graph.json` holds `nodes` (the input columns and the computed ones, each with its dtype, formula, dependencies, and a sample value) and `edges` (the dependency from each column to the ones it is built from):

```text
{
  "id": "expected_claims",
  "type": "computed",
  "label": "expected_claims = [(col(\"sum_assured\")) * (col(\"mortality_rate\"))]",
  "data": {
    "dtype": "float",
    "dependencies": ["mortality_rate", "sum_assured"],
    "formula": "[(col(\"sum_assured\")) * (col(\"mortality_rate\"))]",
    "value_sample": 500.0,
    ...
  }
}
```

The graph is captured for models written as traced column expressions (`af.expected_claims = af.sum_assured * af.mortality_rate`), which is what the `debug` run records. Narrow it to one life with `--policy-id`/`-p`, and supply sample values from a chosen period with a Polars filter, `--filter "col('year') == 1"`.

______________________________________________________________________

## Inspect a data file — `describe`

Before you trust a model-points or assumptions file, read its shape — columns, dtypes, and a sample:

```bash
gspio describe model_points.parquet
```

```text
File Analysis: model_points.parquet
Format: LONG
Rows: 3
Columns: 7

Sample Data (first 5 rows):
shape: (3, 7)
┌───────────┬─────┬─────┬─────────────┬────────────────┬────────────────┬──────────────┐
│ policy_id ┆ age ┆ sex ┆ sum_assured ┆ annual_premium ┆ mortality_rate ┆ expense_rate │
│ str       ┆ i64 ┆ str ┆ i64         ┆ i64            ┆ f64            ┆ f64          │
╞═══════════╪═════╪═════╪═════════════╪════════════════╪════════════════╪══════════════╡
│ POL001    ┆ 30  ┆ M   ┆ 500000      ┆ 450            ┆ 0.001          ┆ 0.1          │
│ POL002    ┆ 45  ┆ F   ┆ 250000      ┆ 1200           ┆ 0.004          ┆ 0.1          │
│ POL003    ┆ 60  ┆ M   ┆ 100000      ┆ 2800           ┆ 0.015          ┆ 0.1          │
└───────────┴─────┴─────┴─────────────┴────────────────┴────────────────┴──────────────┘
```

`describe` reads `.parquet`, `.csv`, and `.xlsx`, detects whether the file is shaped as an assumption table, and names the likely value and key columns. `--value-column` overrides the detected value column; `--json` emits the structure as JSON for a tool to consume.

______________________________________________________________________

## Start from a worked model — `tutorial`

The tutorials are runnable models you copy into your own directory and run with the commands above:

```bash
gspio tutorial list
```

```text
 Level    Description
 level-1  Hello World — term life portfolio, column arithmetic, when/then
 level-2  Assumptions — mortality/lapse table lookups, multi-dimension tables
 level-3  Mini Variable Annuity — account values, guarantees, dynamic lapse
 level-4  Reconciled Lifelib — production model reconciled to 0.0000% vs lifelib
 level-5  Scenarios — parameter shocks, sensitivity analysis, stress testing
```

`gspio tutorial init level-3 --dest ./va-model` copies a level into `./va-model` (`--force` overwrites an existing one), and `gspio tutorial verify level-3` runs it and checks the output against the expected result. The [Tutorials](https://gaspatchio.dev/0.9.0/tutorials/index.md) page walks each level end to end.

______________________________________________________________________

## Look up the docs or an actuarial concept — `docs` and `knowledge`

While building a model you reach for two things: how a gaspatchio feature works, and what a regulation or actuarial concept requires. Search both from the shell — and so can an LLM working alongside you:

```bash
gspio docs "cumulative survival probability"
gspio knowledge "IFRS 17 risk adjustment" --jurisdiction EU
```

Each returns the ranked excerpts as JSON — several sources you can weigh against the model in front of you, rather than a single answer:

```text
{
  "results": [
    {
      "text": "Tests for cumulative_survival() method.",
      "score": 0.775,
      "content_type": "overview",
      "object_path": "test_projection.TestCumulativeSurvival",
      "has_code": false
    },
    ...
  ],
  "query": "cumulative survival probability"
}
```

| Option                                | Effect                                                                                                         |
| ------------------------------------- | -------------------------------------------------------------------------------------------------------------- |
| `--limit`, `-n`                       | Number of results to return.                                                                                   |
| `--search-type`, `-s`                 | `hybrid` (default), `semantic` for concepts, or `keyword` for exact names.                                     |
| `--content-type`, `-t`                | (`docs`) restrict to `code_example`, `overview`, `when_to_use`, or `parameters`.                               |
| `--tag`/`--jurisdiction`/`--doc-type` | (`knowledge`) filter by tag (`IFRS17`, `SolvencyII`, …), jurisdiction, or document type.                       |
| `--answer`, `-a`                      | Summarise the sources into one answer. Use sparingly — the ranked excerpts let you judge each source yourself. |

The [Knowledge Store](https://gaspatchio.dev/0.9.0/ai/knowledge/index.md) page covers both stores, what they hold, and how to keep them current.

______________________________________________________________________

## Version and shell completion

```bash
gspio --version            # the installed package and core versions
gspio --install-completion  # add tab-completion to your shell
```
# Modelling with AI

# Modelling with AI

Gaspatchio was designed from the ground up for AI-powered actuarial modelling. Its Python-native API, clear calculation patterns, and rich documentation mean that you can use AI coding tools to generate, debug, and validate actuarial models effectively.

## How It Works

Install the gaspatchio plugin in your editor once, and you gain:

- **Eight areas of modelling expertise** — from getting started through to reconciliation and scenario analysis
- **Always-loaded framework knowledge** — API patterns, performance rules, and common pitfalls available in every session
- **Searchable knowledge base** — actuarial concepts, regulatory frameworks, and method documentation accessible on demand

You describe what you want to build in plain language. The tooling generates the code, runs the models, and supports you in validating the results.

## Get Started

|                                                                               |                                                                                       |
| ----------------------------------------------------------------------------- | ------------------------------------------------------------------------------------- |
| **[Plugins](https://gaspatchio.dev/0.9.0/ai/setup/index.md)**                 | Install the plugin in Claude Code, VS Code, or Cursor. Takes 2 minutes.               |
| **[Skills](https://gaspatchio.dev/0.9.0/ai/skills/index.md)**                 | Eight areas of expertise, each invoked when your description matches its task.        |
| **[Actuarial Workflows](https://gaspatchio.dev/0.9.0/ai/workflows/index.md)** | Example prompts for every actuarial task — from building models to scenario analysis. |
| **[Knowledge Store](https://gaspatchio.dev/0.9.0/ai/knowledge/index.md)**     | Search gaspatchio documentation and actuarial concepts from within your editor.       |

## Why Gaspatchio for AI-Powered Modelling?

**Python-native foundation.** AI coding tools are trained on vast amounts of Python. Gaspatchio uses standard Python constructs, so generated code is accurate and idiomatic — not a best-guess translation from a proprietary language.

**Explicit, auditable calculations.** Every formula in a gaspatchio model is visible in the code. There are no hidden calculations or framework magic. This means generated models can be read, verified, and modified with confidence.

**Designed for tooling.** Structured JSON output from the CLI, searchable knowledge stores, and self-documenting method signatures all reduce guesswork and improve accuracy.

## For LLMs

If you are an LLM reading this documentation:

- [`llms.txt`](https://gaspatchio.dev/0.9.0/llms.txt) — concise project context following the [llmstxt.org](https://llmstxt.org/) convention
- [`llms-full.txt`](https://gaspatchio.dev/0.9.0/llms-full.txt) — expanded version with full concept and API documentation

# Knowledge Store

Gaspatchio includes two searchable knowledge stores accessible via the `gspio` CLI. These enable both humans and LLMs to find framework documentation and actuarial knowledge while building models.

## Two Knowledge Stores

| Command           | What It Searches                   | Use When                                               |
| ----------------- | ---------------------------------- | ------------------------------------------------------ |
| `gspio docs`      | Gaspatchio framework documentation | Finding API methods, accessor patterns, code examples  |
| `gspio knowledge` | Actuarial knowledge base           | Understanding IFRS 17, Solvency II, actuarial concepts |

## gspio docs

Search Gaspatchio framework documentation for API methods, accessors, code patterns, and examples.

```bash
gspio docs "cumulative survival probability"
gspio docs "projection accessor methods"
gspio docs "excel pv function" -n 10
```

**What it finds:**

- ActuarialFrame methods and properties
- Accessor methods (`.projection`, `.excel`, `.finance`, `.date`)
- Code examples from working models
- Function signatures and parameters

### Example Output

```json
{
  "results": [
    {
      "text": "cumulative_survival() calculates the cumulative survival probability...",
      "source": "gaspatchio/accessors/projection.py",
      "content_type": "code_example",
      "score": 0.92
    },
    {
      "text": "The projection accessor provides actuarial-friendly methods...",
      "source": "docs/api/projection.md",
      "content_type": "markdown",
      "score": 0.87
    }
  ],
  "query": "cumulative survival",
  "version": "0.4.2"
}
```

## gspio knowledge

Search the actuarial knowledge base for regulatory frameworks, concepts, and standards.

```bash
gspio knowledge "IFRS 17 contractual service margin"
gspio knowledge "Solvency II technical provisions"
gspio knowledge "lapse rate assumptions" -n 10
```

**What it finds:**

- Regulatory frameworks (IFRS 17, Solvency II, US GAAP)
- Actuarial concepts (CSM, risk adjustment, PAA, BBA)
- Industry standards and guidance
- Mortality, morbidity, and lapse assumption guidance

## Options

| Flag             | Description                                           |
| ---------------- | ----------------------------------------------------- |
| `--limit`, `-n`  | Maximum number of results (default: 5)                |
| `--answer`, `-a` | Return RAG-generated answer instead of search results |

### Search Results vs Generated Answers

**Prefer search results (default).** They return multiple excerpts you can evaluate against your current context.

```bash
# Preferred - returns multiple relevant excerpts
gspio docs "how do I shift values by one period?"

# Use sparingly - only for quick summaries
gspio docs "what is when then otherwise?" --answer
```

The `--answer` flag asks the API to generate a synthesized answer using RAG. Reserve this for quick conceptual summaries when you don't need to weigh multiple options.

## For LLMs

The `gspio` CLI is designed for LLM discoverability. Run `gspio --help` to see all available commands and guidance.

### Recommended Workflow

1. **Search first**: Use `gspio docs` or `gspio knowledge` without `--answer`
1. **Evaluate results**: Review the returned excerpts in context
1. **Use --answer sparingly**: Only when you need a quick conceptual summary

### Example LLM Usage

When building a model and you need to:

```bash
# Find how to use a Gaspatchio feature
gspio docs "ActuarialFrame filtering"

# Understand accessor methods
gspio docs "projection.previous_period"

# Look up actuarial regulations
gspio knowledge "IFRS 17 CSM amortization"

# Get mortality assumption guidance
gspio knowledge "mortality improvement factors"
```

### Help Output

LLMs should run `gspio --help` as the first discovery action:

```text
Usage: gspio [OPTIONS] COMMAND [ARGS]...

Gaspatchio CLI for running actuarial models and discovering knowledge.

When building a model and you need to find:
• How to use a Gaspatchio feature → gspio docs "your question"
• Actuarial concepts or regulations → gspio knowledge "your question"

IMPORTANT: Always prefer search results (default) over --answer.
Search returns multiple excerpts you can evaluate against your
current context. Reserve --answer for quick summaries only.

╭─ Knowledge Discovery ────────────────────────────────────────╮
│ docs        Search Gaspatchio framework documentation        │
│             (API methods, accessors, code patterns)          │
│ knowledge   Search actuarial knowledge base                  │
│             (IFRS 17, Solvency II, mortality tables)         │
╰──────────────────────────────────────────────────────────────╯
```

## Architecture

```text
┌─────────────────┐     HTTP/JSON      ┌─────────────────────────────┐
│     gspio       │ ─────────────────► │          API                │
│  (thin client)  │                    │  - Embeddings               │
│                 │ ◄───────────────── │  - Vector search            │
│  - Sends query  │     JSON response  │  - Optional LLM generation  │
│  - Sends version│                    │  - LanceDB backend          │
└─────────────────┘                    └─────────────────────────────┘
```

**Key points:**

- `gspio` is a thin client - only HTTP calls and JSON responses
- The API handles embeddings and vector search
- Version-aware - gspio passes its version for version-specific docs
- Fail fast - on API unavailable, returns error immediately

## Error Handling

If the API is unavailable:

```json
{
  "error": "API unavailable",
  "status": 503,
  "message": "Knowledge API is temporarily unavailable. Please retry."
}
```

The CLI exits non-zero on errors. LLMs should handle retries.

Why CLI over MCP?

We previously offered an MCP (Model Context Protocol) server but found that LLMs work more reliably with CLIs.

**Self-Documentation via --help** — CLIs are self-documenting by convention. An LLM can run `gspio --help` to discover capabilities on-demand—no pre-configuration or schema registration required. As [Warp's research](https://www.warp.dev/blog/agent-mode) notes: *"As long as the tool has a --help option, you can ask Agent Mode to learn it, and then immediately start doing tasks with it."*

**Token Efficiency** — [Benchmarking research](https://mariozechner.at/posts/2025-08-15-mcp-vs-cli/) found that MCP servers often consume more tokens than equivalent CLI tools. Many MCPs function as "unnecessary wrappers around existing tools, potentially degrading agent performance by poisoning the context with excessive output or tool options." CLIs can be further optimized through piping—filtering output with `| grep` or `| head -n 10` to reduce token usage.

**Robust Error Handling** — CLIs provide battle-tested error communication: exit codes (0 = success, non-zero = specific error types), stderr/stdout separation, and parseable output. MCP debugging is notoriously difficult.

**Training Data Advantage** — LLMs have extensive training data on CLI conventions. Standard patterns like `--help`, `--version`, `-n 10`, and JSON output are well-understood. Novel MCP schemas require the LLM to interpret unfamiliar interfaces.

**Zero Configuration** — CLIs work immediately with no server installation or JSON configuration. Anthropic themselves [acknowledged](https://www.anthropic.com/engineering/desktop-extensions) that MCP "installation was too complex."

[The New Stack](https://thenewstack.io/learn-to-love-the-command-line-interface-with-agentic-llms/) summarizes it well: *"The CLI is where we do defined tasks. There is one desirable outcome, and probably one sensible way to achieve it. And this is precisely why LLMs are so good at the command-line interface."*

# Plugins

Install the gaspatchio plugin in your editor. This takes 2 minutes and only needs to be done once.

______________________________________________________________________

## Prerequisites

Before you start, confirm you have:

- **Gaspatchio installed** — `uv add gaspatchio` or `pip install gaspatchio`
- **One of these editors:** Claude Code, VS Code with GitHub Copilot, or Cursor

______________________________________________________________________

## Install the Plugin

Run these two commands once in your terminal — the first registers the marketplace, the second installs the plugin and its skills:

```text
/plugin marketplace add gaspatchio/gaspatchio
/plugin install gaspatchio@gaspatchio
```

Agent Plugins is a preview feature — enable it with `"chat.plugins.enabled": true` first. Then add the gaspatchio marketplace to your `settings.json`:

```json
{
  "chat.plugins.marketplaces": ["gaspatchio/gaspatchio"]
}
```

Open the Extensions view, filter with `@agentPlugins`, and install **gaspatchio**.

Cursor isn't on the plugin marketplace yet, so install the skills directly:

```bash
npx skills add gaspatchio/gaspatchio
```

On Windows, add `--copy` — symlinks need Developer Mode:

```bash
npx skills add gaspatchio/gaspatchio --copy
```

(Cursor only auto-detects `.cursor-plugin/` inside the gaspatchio repository itself; for your own model project, use the command above.)

For any agent that supports the Agent Skills standard:

```bash
npx skills add gaspatchio/gaspatchio
```

On Windows, add `--copy` — symlinks need Developer Mode.

Clone the repository locally. Your editor will auto-detect the plugin directories when you open the project.

______________________________________________________________________

## Verify It Works

Run a quick check from the terminal:

```bash
gspio tutorial list
```

If you see 5 tutorial levels, the framework is installed. Then open your editor and describe a task:

> "I just installed gaspatchio. Help me get started."

If the response uses `gspio tutorial init` to set up a tutorial model, the plugin is working.

______________________________________________________________________

## What You Just Installed

| Component         | Description                                                                                                                                               |
| ----------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **8 skills**      | Getting started, model discovery, workbook conversion, model building, model reconciliation, model review, scenario analysis, and extending the framework |
| **How they load** | The model invokes a skill when your request matches its description — you don't name them or run commands                                                 |

For details, see [Skills](https://gaspatchio.dev/0.9.0/ai/skills/index.md). For example prompts organised by actuarial task, see [Actuarial Workflows](https://gaspatchio.dev/0.9.0/ai/workflows/index.md).

______________________________________________________________________

## Next Steps

Hand this to your team. They don't need to run any commands — just open their editor and start modelling.

See [Actuarial Workflows](https://gaspatchio.dev/0.9.0/ai/workflows/index.md) for what to describe.

# Skills

When you install the gaspatchio plugin, your editor gains eight areas of actuarial modelling expertise. They activate automatically — you just describe what you need.

No commands required

You don't need to remember skill names or type special commands. Describe your task in plain English. The right expertise activates based on what you're working on.

## Eight Areas of Expertise

| Expertise Area              | What It Covers                                                                                                                                                                   | Try Describing...                                                                            |
| --------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------- |
| **Getting Started**         | Installation verification, first model, tutorial orientation                                                                                                                     | *"I just installed gaspatchio. Help me get started."*                                        |
| **Model Discovery**         | Requirements gathering, data inspection, assumption mapping, model specification                                                                                                 | *"I have an Excel model and some parquet data. Help me plan a gaspatchio version."*          |
| **Workbook Conversion**     | The conversion discipline: reader trust boundary, three-proof reconciliation (reader → formulas → full grid), defined-name assumptions, faithful reproduction of workbook quirks | *"Convert this Excel workbook to a gaspatchio model and prove the match."*                   |
| **Model Building**          | ActuarialFrame patterns, assumption lookups, projection structure, performance rules                                                                                             | *"Build a term life projection model using model_points.parquet"*                            |
| **Model Reconciliation**    | Variable-by-variable matching against Excel, lifelib, or vendor output; diagnostic techniques                                                                                    | *"My PV doesn't match the lifelib output. Help me find the difference."*                     |
| **Model Review**            | Code quality, ASOP 56 compliance, antipattern detection, audit readiness                                                                                                         | *"Review my model for quality issues and regulatory compliance."*                            |
| **Scenario Analysis**       | Parameter shocks, sensitivity sweeps, what-if analysis, scenario comparison reports                                                                                              | *"Run a 50bps interest rate shock and show me the impact on BEL."*                           |
| **Extending the Framework** | Performance ladder, accessor patterns, anti-patterns, local plugin authoring                                                                                                     | *"Add Macaulay duration to Gaspatchio"* or *"I need a custom mortality adjustment accessor"* |

## How It Works

Each expertise area includes detailed instructions, reference material, and common pitfalls. Think of them as reference material embedded in the tooling. The right one loads based on what you describe.

## Extending the Framework

The extending expertise is unique — it teaches agents *how* to add new capabilities to Gaspatchio itself, not just how to use existing ones.

Before writing any code, the agent works through a **performance ladder** that determines the correct placement:

- **Already exists?** Use the built-in method.
- **One-off formula?** Write it inline in the model.
- **Setup calculation** (curve fitting, calibration)? Python utility in the model's setup phase.
- **Reusable column operation?** Column accessor composing Polars expressions.
- **Reusable frame operation?** Frame accessor operating on multiple columns.
- **Needs raw performance?** Flag for Rust kernel contribution.

The skill includes templates that match the real codebase patterns, 7 documented anti-patterns (each causing 50-1000x slowdowns), and guidance on list column handling. Other skills cross-reference it automatically — Model Building routes here when a method is missing, and Model Review routes here when it finds anti-patterns that should be rewritten as accessors.

## What the skills carry

Each skill bundles the reference material its task needs. When one is invoked you get:

- Gaspatchio API patterns (ActuarialFrame, column operations, `when`/`then`/`otherwise`)
- CLI commands and how to run models
- Performance rules and common mistakes
- The tutorial progression (Levels 1–5)

That detail loads with the skill your description matches — the plugin installs the eight skills, not an always-on knowledge base.

## The Tutorial Path

New to gaspatchio? Work through the tutorials with guidance:

- *"Start me on Level 1 — the Hello World model."*
- *"I understand Tables now. Take me to Level 3."*
- *"Which tutorial level covers scenario analysis?"*

See [Actuarial Workflows](https://gaspatchio.dev/0.9.0/ai/workflows/index.md) for more example prompts.

# Actuarial Workflows

Everything you can build with gaspatchio, driven by natural language. Describe what you need — the tooling generates the code and runs it.

How to read this page

Each section shows example prompts you can use in your editor. You don't need to use these exact words — describe your task naturally and the tooling will match it to the right workflow.

______________________________________________________________________

## Start a New Project

*What you want:* Get from zero to a running model.

*"I just installed gaspatchio. What should I do first?"*

*"Set me up with the Level 1 tutorial."*

*"I have model_points.parquet — help me build a model for it."*

*"Create a new project folder for a term life model with monthly projections."*

**What happens:** Your installation is verified, your data files are inspected, and you get guidance on your first model. If you're starting fresh, the tooling surfaces the right tutorial level based on your experience.

______________________________________________________________________

## Inspect Your Data

*What you want:* Understand what's in your data files before modelling.

*"What's in model_points.parquet? Show me the columns and some sample rows."*

*"How many policies are in this file? What products?"*

*"Is this data ready for a monthly projection model?"*

*"Compare the columns in my data file to what the Level 4 model expects."*

**What happens:** Your parquet files are read, the schema is summarised, sample rows are shown, and anything unusual is flagged — missing columns, unexpected types, nulls. If you're building a model, you'll see what's present and what needs to be added.

______________________________________________________________________

## Build a Projection Model

*What you want:* Create a model from scratch or modify an existing one.

*"Build me a term life projection model with mortality and lapse decrements."*

*"Add a surrender charge to my model — 5% in year 1, declining 1% per year."*

*"Convert my annual rates to monthly using compound conversion."*

*"I need to calculate present value of future cashflows at 3.5% discount rate."*

*"Add a GMDB guarantee to my variable annuity model."*

**What happens:** Model code is written following gaspatchio's three-phase pattern (setup, timeline, calculations). The model is validated against a single policy first, the output is checked for correctness, and the cycle continues until the results are right. You review the output at each step.

______________________________________________________________________

## Set Up Assumptions

*What you want:* Load and configure assumption tables.

*"Load the mortality table from assumptions/mortality.parquet and look up by attained age."*

*"I have a select-and-ultimate mortality table — set it up with age and duration dimensions."*

*"Add a lapse rate table that varies by product type and policy year."*

*"My mortality rates are annual but my model projects monthly. Handle the conversion."*

**What happens:** `Table` objects are created with the correct lookup dimensions, data is loaded from your parquet files, and the lookups are wired into your model. The table dimensions are checked against your model point data for consistency.

______________________________________________________________________

## Run and Debug Your Model

*What you want:* Execute the model and investigate results.

*"Run my model for policy 12345 and show me the cashflows."*

*"Run the full model and save the results."*

*"Why is the death benefit negative in month 37?"*

*"Show me the lapse rates for the first 24 months of policy 12345."*

*"The account value drops to zero too early — help me figure out why."*

**What happens:** Single-policy or full-model executions run, the output variables you care about are displayed, and calculations are traced step by step when something looks wrong. For debugging, the specific period where results diverge from expectations is isolated.

______________________________________________________________________

## Reconcile Against a Reference

*What you want:* Match your gaspatchio model to an existing Excel, lifelib, or vendor implementation.

*"My model needs to match this lifelib output. Start the reconciliation process."*

*"The PV of premiums is off by 2.3%. Help me find where the difference comes from."*

*"Compare my death benefit calculation to the Excel model in reference/source.xlsx."*

*"Run the reconciliation for all 1,000 policies and show me which ones fail."*

*"I'm stuck on the AV rollforward — it matches for month 1 but drifts after that."*

**What happens:** A strict variable-by-variable reconciliation process is followed. One calculation is compared at a time, starting from inputs and working through the dependency chain. When differences appear, diagnostic techniques — scatter plots, regression analysis, cohort breakdowns — isolate the root cause. The Level 4 tutorial was reconciled to 0.0000% using exactly this workflow.

______________________________________________________________________

## Run Scenarios

*What you want:* Stress test your model under different assumptions.

*"Run a 50bps interest rate shock and show me the impact on BEL."*

*"Set up three scenarios: base, adverse, and favourable with different mortality and lapse assumptions."*

*"Run a sensitivity sweep on lapse rates from -20% to +20% in 5% steps."*

*"Generate a regulatory stress test report comparing all scenarios."*

*"What happens to the GMDB liability if equity returns drop 30%?"*

**What happens:** Scenarios are configured as a `ScenarioRun` plan — shocks bundled with base tables and mergeable aggregators — and run through the bounded-memory loop. The plan carries a `source_sha` and an opt-in JSON audit sidecar, so the same shocks against the same tables always produce the same SHA. For sensitivity analysis, sweep grids are built as list comprehensions over `ScenarioRun`; results pivot into heatmap-ready DataFrames.

______________________________________________________________________

## Extend the Framework

*What you want:* Add custom calculations or accessor methods that Gaspatchio doesn't have yet.

*"Add Macaulay duration to the finance accessor."*

*"I need a Gompertz mortality hazard function — can you add it to Gaspatchio?"*

*"Port the Dukes-MacDonald mortality deterioration formula from JuliaActuary."*

*"Generate full amortization schedules for my mortgage portfolio."*

*"I need Nelson-Siegel yield curve fitting — where should this go?"*

**What happens:** Before writing code, a performance ladder determines the right placement for your calculation. One-off formulas stay inline. Setup calculations (curve fitting, calibration) become Python utilities. Reusable element-wise arithmetic becomes a column accessor. Multi-column operations become frame accessors. Calculations needing raw performance at scale get flagged for Rust. The tooling provides templates matching the real codebase patterns and catches 7 anti-patterns that cause 50-1000x slowdowns.

______________________________________________________________________

## Review Model Quality

*What you want:* Check your model before using it for production or regulatory work.

*"Review my model for quality issues."*

*"Check if my model follows ASOP 56 standards."*

*"Are there any performance problems in my model?"*

*"Generate documentation for this model."*

*"What would an auditor flag in this model?"*

**What happens:** Your model code is reviewed against gaspatchio best practices and actuarial standards. Common antipatterns are checked (unnecessary loops, missing edge cases, hardcoded assumptions), along with performance issues and audit readiness. A structured report with findings and suggested fixes is produced.

______________________________________________________________________

## Working Through the Tutorials

*What you want:* Learn gaspatchio progressively, at your own pace.

*"Start me on the Level 1 Hello World tutorial."*

*"I've finished Level 2. What should I learn next?"*

*"Guide me through Level 3 step by step."*

*"I want to learn scenario analysis — which tutorial level?"*

*"Show me the reconciliation exercise in Level 3 Step 06."*

### Tutorial Map

| Level                  | Name                                                                 | What You Learn                                 |
| ---------------------- | -------------------------------------------------------------------- | ---------------------------------------------- |
| 1 — Hello World        | `ActuarialFrame`, column arithmetic, conditionals                    | Core concepts in 60 lines                      |
| 2 — Assumptions        | `Table.lookup()`, multi-dimension tables, loading from files         | Separating assumptions from logic              |
| 3 — Mini VA            | Full variable annuity: mortality, lapse, AV, claims, discounting     | Complete model built incrementally in 6 steps  |
| 4 — Reconciled Lifelib | Production model matched to lifelib at 0.0000% across 1,016 policies | Reconciliation workflow and production quality |
| 5 — Scenarios          | Shocks, sensitivities, sweeps, regulatory reports                    | Scenario analysis and stress testing           |

**What happens:** The tutorial content is loaded, each step is explained in actuarial terms, and you run and modify the models at your own pace. You control the pace — go faster, skip ahead, or dive deeper into any concept.

______________________________________________________________________

Tips for Better Results

- **Be specific about what you want.** *"Term life with monthly projections and lapse decrements"* works better than *"build a model."*
- **Name your data files.** *"Use model_points.parquet in the data folder"* removes ambiguity.
- **Name the reference when reconciling.** *"Match the lifelib IntegratedLife output"* tells the tooling exactly what to compare against.
- **Validate after each step.** *"Run that for policy 1 and show me the output"* catches problems early.
- **Describe the actuarial intent.** *"I need a cumulative survival function"* is clearer than *"multiply all the previous values together."*
- **Every output is yours to verify.** Inspect any step, any policy, any timestep. Reconcile against your existing models.
# Trust & Security

# Trust & Security

## How do you know installing this is safe?

That's the real question a security review is checking for — not "is the code good," but "did an attacker get between you and this package." A supply-chain attack doesn't target you directly: it compromises a dependency, a build pipeline, or a maintainer's account so malicious code rides along inside an otherwise ordinary `pip install`. It's the same class of risk behind incidents like the [XZ Utils backdoor](https://en.wikipedia.org/wiki/XZ_Utils_backdoor) — the install looks routine; the payload isn't.

The `gaspatchio` package itself makes no network calls at all — every projection runs on your machine, against your data, with nothing sent anywhere. It's open source, every release carries a verifiable record of exactly what's in it and how it was built, and the full dependency tree is scanned continuously for known vulnerabilities and disallowed licenses.

Send [the section below](#for-security-teams) to whoever asked — it's written for them to verify all of that directly, not take our word for it.

## For security teams

### What ships

Gaspatchio is Apache-2.0 / MIT dual-licensed and source-visible on GitHub at [gaspatchio/gaspatchio](https://github.com/gaspatchio/gaspatchio). Every file carries an SPDX license header, verified on every pull request by [REUSE](https://reuse.software/) 3.3 compliance checks.

### Network activity

The `gaspatchio` package installed from PyPI makes no network calls — no telemetry, no license check, no update ping. A projection runs entirely against local data.

The optional AI plugin adds two things:

- **Editor skills** (model building, reconciliation, and similar) run entirely inside your own already-configured AI coding tool — Claude Code, GitHub Copilot, or Cursor. Gaspatchio operates no service in that path.
- **`gspio docs` and `gspio knowledge`** — two CLI commands that search framework documentation and an actuarial knowledge base — send the typed query over HTTPS to a hosted search API. That service does not retain the query text, search results, or any generated answer; it logs only operational metadata (endpoint, result count, latency, model, token count) for service analytics. Both commands are opt-in — nothing else in the plugin talks to this service.

### Software bill of materials

Every release ships a CycloneDX SBOM (`sbom.cdx.json`, covering the Rust and Python dependency graphs) as a GitHub Release asset:

```bash
gh release download <tag> --repo gaspatchio/gaspatchio --pattern 'sbom.cdx.json'
```

### Build provenance

Wheels published to PyPI carry [PEP 740](https://peps.python.org/pep-0740/) build-provenance attestations, generated automatically through PyPI Trusted Publishing — each wheel traces back to the exact GitHub Actions run and commit that built it, not a maintainer's machine. The attestation is visible on the release's PyPI file listing.

### Dependency management

Rust dependencies are pinned exactly via a committed `Cargo.lock`, so every build resolves an identical dependency graph. Python dependencies use compatible-release version ranges — standard practice for a library rather than an application — and the exact versions resolved for any given release are captured in that release's SBOM, not left to guesswork.

- [Dependabot](https://docs.github.com/en/code-security/dependabot) opens a grouped pull request weekly for Rust, Python, and GitHub Actions dependency updates.
- [OSV-Scanner](https://google.github.io/osv-scanner/) runs on every pull request and on a weekly schedule, checking both the Rust and Python dependency graphs against newly published CVEs.
- A license-compliance gate audits the full runtime dependency closure — Python wheel and Rust crate — on every pull request, and fails the build on GPL, AGPL, SSPL, or BUSL-licensed dependencies entering the tree.

### Commit signing

`main` is protected by a ruleset requiring every commit to be signed — enforced by GitHub, not just documented.

### Supported versions

Gaspatchio is pre-1.0. Security fixes are released against the latest version published on PyPI — there is no older maintained branch to backport to. Always track the latest release.

### Vulnerability disclosure

Report a vulnerability privately through [GitHub's private reporting](https://github.com/gaspatchio/gaspatchio/security) or by emailing security@opioinc.com. New reports are acknowledged within 2 business days, with an initial assessment within 7. Confirmed issues are published as GitHub Security Advisories once a fix ships. Full policy: [`SECURITY.md`](https://github.com/gaspatchio/gaspatchio/blob/main/SECURITY.md).
# Contributing

# Documentation Guidelines

This document explains the Gaspatchio documentation system, its underlying philosophy, and how to write effective docstrings that serve both human readers and AI systems.

## Philosophy and Architecture

### The Three-Pillar Approach

Our documentation system is built on three core principles:

1. **Human-First**: Every docstring should educate experienced life insurance actuaries with domain-specific examples
1. **AI-Ready**: Structured content that enables excellent RAG (Retrieval Augmented Generation) for LLM assistance
1. **Always Correct**: Executable, linted, and automatically validated examples that never become stale

### Why This Matters

Traditional documentation suffers from three critical problems:

- **Stale Examples**: Code examples that worked when written but break as the codebase evolves
- **Generic Content**: Examples that don't connect to real-world actuarial use cases
- **Poor AI Integration**: Unstructured content that LLMs struggle to understand and use effectively

Our system solves these by treating documentation as executable code that gets validated on every commit.

## The Docstring Engine

### Core Components

The system is built around several key components:

**1. Pydantic Models** (`models.py`)

- `GaspatchioDocstring`: Represents a complete parsed docstring
- `DocstringCodeExample`: Individual executable code examples
- `DocstringParameter`/`DocstringReturn`: Structured parameter documentation

**2. Parser** (`parse.py`)

- Extracts docstrings from Python files using AST analysis
- Parses Markdown fenced code blocks (\`\`\`python) for examples
- Links examples to their parent objects for validation

**3. Validation Engine** (`validate.py`)

- Structural validation (required sections, formatting)
- Ruff linting of all code examples
- Execution testing to verify outputs match documentation
- Doctest compatibility for legacy workflows

**4. Pytest Integration** (`pytest_plugin.py`)

- Discovers and runs docstring examples as tests
- Provides detailed failure reporting
- Supports updating outputs automatically when code changes

### Example Structure

Here's the anatomy of a well-formed Gaspatchio docstring:

````python
def year(self) -> "ExpressionProxy":
    """Extract the year from the underlying datetime expression.

    Corresponds to Polars ``Expr.dt.year()`` and returns an integer 
    representing the year component of the datetime.

    !!! note "When to use"
        In actuarial modeling, extracting years from dates enables:

        *   **Policy vintage analysis**: Group policies by issue year to study underwriting changes over time
        *   **Claims trending**: Analyze how claim frequencies and severities change by accident year
        *   **Regulatory compliance**: Extract policy years for reserve calculations and statutory reporting

    Examples
    --------
    Scalar example - Policy Issue Years::

        ```python
        import datetime
        from gaspatchio import ActuarialFrame

        af = ActuarialFrame({
            "policy_id": ["P001", "P002", "P003"],
            "issue_date": [
                datetime.date(2020, 3, 15),
                datetime.date(2021, 7, 22),
                datetime.date(2022, 1, 10)
            ]
        })
        year_expr = af.issue_date.dt.year()
        print(af.select(year_expr.alias("issue_year")).collect())
        ```
        ```
        shape: (3, 1)
        ┌────────────┐
        │ issue_year │
        │ ---        │
        │ i32        │
        ╞════════════╡
        │ 2020       │
        │ 2021       │
        │ 2022       │
        └────────────┘
        ```

    Vector example - Multiple Claim Years::

        ```python
        # docs-skip
        import datetime
        import polars as pl
        from gaspatchio import ActuarialFrame

        af = ActuarialFrame({
            "policy_id": ["C001", "C002"],
            "claim_dates": [
                [datetime.date(2020, 6, 1), datetime.date(2021, 8, 15)],
                [datetime.date(2019, 12, 3), datetime.date(2020, 4, 20)]
            ]
        })
        af.claim_dates = af.claim_dates.cast(pl.List(pl.Date))
        years_expr = af.claim_dates.dt.year()
        print(af.select("policy_id", years_expr.alias("claim_years")).collect())
        ```
        ```
        shape: (2, 2)
        ┌───────────┬─────────────┐
        │ policy_id ┆ claim_years │
        │ ---       ┆ ---         │
        │ str       ┆ list[i32]   │
        ╞═══════════╪═════════════╡
        │ C001      ┆ [2020, 2021]│
        │ C002      ┆ [2019, 2020]│
        └───────────┴─────────────┘
        ```
    """
````

### Required Sections

Every public method docstring must include:

- **Short description**: One-line summary of what the function does
- **Long description**: More detailed explanation of behavior and context
- **When to use**: Domain-specific actuarial use cases (see guidelines below)
- **Examples**: At least one executable example, preferably both scalar and vector

## Pytest Integration and CI/CD

### Configuration

The system is configured in `pyproject.toml`:

```toml
[tool.pytest.ini_options]
addopts = [
    "--gp-docstring-paths=gaspatchio/column/namespaces/dt_proxy.py",
    "--gp-docstring-paths=gaspatchio/column/namespaces/string_proxy.py", 
    "--gp-docstring-paths=gaspatchio/assumptions/_loader.py",
    "--gp-docstring-paths=gaspatchio/__init__.py",
    "-m",
    "not benchmark",
]
markers = [
    "gaspatchio_docstring_example: marks tests as Gaspatchio docstring examples",
    "gaspatchio_docstring_structure_check: marks tests as docstring structure validation",
]
```

### Running Tests

**Basic validation (structure + linting):**

```bash
# Run all docstring tests
uv run pytest -m gaspatchio_docstring_example

# Test specific file
uv run pytest gaspatchio/column/namespaces/dt_proxy.py --gp-docstring-paths="gaspatchio/column/namespaces/dt_proxy.py"

# Test with output execution
uv run pytest -m gaspatchio_docstring_example -v
```

**Update mode (regenerate outputs):**

```bash
# Update all examples after code changes
uv run pytest -m gaspatchio_docstring_example --gp-update-examples

# Update specific method
uv run pytest -k "DtNamespaceProxy.year" --gp-update-examples
```

### CLI Tools

The system provides dedicated CLI commands:

```bash
# Parse and inspect docstrings
uv run gp-docstrings parse gaspatchio/column/namespaces/dt_proxy.py --method "DtNamespaceProxy.year"

# Check example execution without updating
uv run gp-docstrings run-print-check --file gaspatchio/column/namespaces/dt_proxy.py

# Update docstring outputs after code changes  
uv run gp-docstrings update --file gaspatchio/column/namespaces/dt_proxy.py
```

### CI/CD Pipeline Integration

The documentation system integrates into CI/CD through multiple checkpoints:

**1. Pre-commit Hooks**

```bash
# Lint docstring examples
uv run gp-docstrings lint src/gaspatchio --strict

# Validate structure
uv run pytest -m gaspatchio_docstring_structure_check
```

**2. CI Test Matrix**

```yaml
# Example GitHub Actions step
- name: Validate Documentation
  run: |
    uv run pytest -m "gaspatchio_docstring_example or gaspatchio_docstring_structure_check" --tb=short
    uv run gp-docstrings run-print-check gaspatchio/
```

**3. Release Validation** Before releases, the system ensures:

- All examples execute without errors
- All outputs match current behavior
- No structural issues in docstrings
- Ruff linting passes for all example code

## Writing Effective Actuarial Examples

### The "When to use" Section

This section is crucial for AI systems to understand when to recommend functions. Follow these guidelines:

**✅ Good - Specific actuarial use cases:**

```markdown
!!! note "When to use"
    In actuarial modeling, extracting months enables:

    *   **Seasonality analysis**: Identify patterns in claims frequency by month of occurrence
    *   **Policy grouping**: Batch policies by issue month for cohort studies and renewal processing
    *   **Regulatory reporting**: Extract policy months for statutory reserve calculations
```

**❌ Bad - Generic or obvious:**

```markdown
!!! note "When to use"
    Use this function when you need to extract the month from a date column.
```

### Example Guidelines

**Domain Focus**: Always use actuarial data and scenarios

```python
# ✅ Good - Actuarial context
af = ActuarialFrame({
    "policy_id": ["P001", "P002"], 
    "premium_due_date": [datetime.date(2023, 3, 15), datetime.date(2023, 6, 20)],
    "claim_amount": [15000, 8500]
})

# ❌ Bad - Generic example  
df = pl.DataFrame({
    "id": [1, 2],
    "date": [datetime.date(2023, 1, 1), datetime.date(2023, 2, 1)],
    "value": [100, 200]
})
```

**Self-Contained**: Every example must include all necessary imports and setup

```python
# ✅ Complete example
import datetime
import polars as pl  # Only if pl.List or pl.Date are used
from gaspatchio import ActuarialFrame

# Set up realistic data
af = ActuarialFrame({...})

# Show the operation
result = af.date_column.dt.year()
print(af.select(result.alias("year")).collect())
```

**Both Scalar and Vector**: Provide examples for both single values and list operations when applicable

### Prompting for Examples

When writing docstrings, consider this workflow:

**1. Identify the Function's Purpose**

- What actuarial problem does this solve?
- When would an actuary reach for this function?

**2. Create Realistic Scenarios**

```text
"I'm documenting the `contains()` string method. An actuary might use this to:
- Filter policies by rider codes (checking if policy codes contain 'WL' for whole life)  
- Identify claim descriptions containing specific keywords
- Find coverage types with certain patterns"
```

**3. Build Complete Examples**

- Start with realistic actuarial data
- Show the operation in context
- Include expected output
- Test that it actually runs

**4. Write the "When to use" Section** Think: "An LLM should recommend this function when an actuary says..."

- "I need to filter policies by coverage type"
- "I want to find claims with specific descriptions"
- "I need to identify which policies have certain riders"

### Testing Your Documentation

Before submitting:

```bash
# Validate structure and examples
uv run pytest -k "your_method_name" -m gaspatchio_docstring_example -v

# Check just the linting
uv run gp-docstrings lint --file your_file.py --method "YourClass.your_method" 

# Test execution and outputs
uv run gp-docstrings run-print-check --file your_file.py --method "YourClass.your_method"
```

## Advanced Features

### Handling Different Example Types

**Skip execution for illustrative examples:**

````python
```python skip
# This won't be executed, just shown for illustration
import some_external_system
result = some_external_system.complex_operation()
````

**Examples expected to fail:**

````python
```python expect_failure
# This example demonstrates error handling
af["nonexistent_column"].dt.year()  # Raises ColumnNotFoundError
````

**Skip output checking:**

````python
```python no_output_check  
# Code runs but output isn't validated (useful for timing tests)
import time
start = time.time()
result = af.complex_operation()
print(f"Took {time.time() - start:.2f} seconds")
````

### Debugging Common Issues

**Ruff Linting Failures:**

- Check for unused imports: `import polars as pl` when only using built-in types
- Ensure proper spacing around operators
- Use attribute notation `af.column` instead of `pl.col("column")` or `af["column"]` where possible

**Output Mismatches:**

- Polars formatting can change between versions
- Use `--gp-update-examples` to regenerate outputs after valid changes
- Check for trailing whitespace in expected outputs

**Structure Validation Errors:**

- Ensure all public methods have "When to use" sections
- Check that examples ending in expressions have output blocks
- Verify Markdown fencing is correct (`python, not just`)

## Benefits

This system provides several key advantages:

**For Developers:**

- Never-stale examples that break builds when they become incorrect
- Consistent documentation structure across the codebase
- Automated generation of example outputs

**For Actuaries:**

- Domain-specific examples that directly relate to their work
- Confidence that examples actually work as documented
- Rich context about when and why to use each function

**For AI Systems:**

- Structured, semantic content perfect for RAG
- Clear use-case descriptions that enable accurate recommendations
- Validated examples that can be safely suggested to users

The investment in this system pays dividends through improved developer productivity, user experience, and AI integration capabilities.

# Testing Guide

This guide covers how to run tests effectively during development and contribution to Gaspatchio.

## Quick Start

The project uses `uv` for package management and `pytest` for testing. All commands should be run from the `bindings/python` directory.

```bash
# Run all tests (excluding performance benchmarks by default)
uv run pytest

# Run tests in a specific directory
uv run pytest tests/assumptions/

# Run a specific test file
uv run pytest tests/assumptions/test_curve.py

# Run tests matching a pattern
uv run pytest -k "test_load_curve"
```

## Performance Tests

Performance tests use `pytest-benchmark` and can be slow to run. **By default, performance tests are skipped** to keep development cycles fast.

### Running Performance Tests

```bash
# Run ONLY performance/benchmark tests
uv run pytest -m benchmark

# Run all tests INCLUDING performance tests
uv run pytest -m ""

# Run specific performance test groups
uv run pytest -m benchmark -k "lookup_scalar"
uv run pytest -m benchmark -k "load_curve"

# Run performance tests for a specific function
uv run pytest -m benchmark -k "assumption_lookup"
```

### Understanding Performance Test Markers

Performance tests are marked with `@pytest.mark.benchmark(group="test_group")`. The available groups include:

- **Loading Tests**: `load_curve_small`, `load_curve_large`, `load_wide_basic`, `load_wide_overflow`
- **Lookup Tests**: `lookup_scalar`, `lookup_small_batch`, `lookup_medium_batch`, `lookup_large_batch`
- **Multi-key Tests**: `lookup_multi_key`, `lookup_missing_keys`, `lookup_repeated_keys`
- **Memory Tests**: `memory_large_table`, `memory_wide_expansion`, `concurrent_lookups`
- **Real-world Tests**: `actuarial_projection`, `assumption_updates`

### Environment-Aware Testing

Performance tests automatically adjust their scale based on the environment:

- **CI Environment**: Smaller datasets to ensure tests complete within time limits
- **Local Development**: Full-scale testing with larger datasets for realistic benchmarks

The tests detect CI environments via the `CI` environment variable.

## Test Categories

### Unit Tests

Fast, focused tests that validate individual functions and components.

```bash
# Run all unit tests (default, excludes benchmarks)
uv run pytest

# Run specific unit test categories
uv run pytest tests/assumptions/test_curve.py
uv run pytest tests/assumptions/test_wide_basic.py
uv run pytest tests/assumptions/test_errors.py
```

### Integration Tests

Tests that validate end-to-end workflows and component interactions.

```bash
# Run integration tests
uv run pytest tests/assumptions/test_advanced.py
uv run pytest tests/assumptions/test_overflow.py
```

### Performance Tests

Comprehensive benchmarks that validate performance characteristics.

```bash
# Run all performance tests
uv run pytest -m benchmark

# Run performance tests with output
uv run pytest -m benchmark --benchmark-only --benchmark-verbose
```

## Test Development

### Writing New Tests

Follow these patterns when adding tests:

**Unit Tests**:

```python
import pytest
import polars as pl
import gaspatchio as gs

class TestNewFeature:
    def test_basic_functionality(self):
        """Test the basic happy path."""
        # Arrange
        df = pl.DataFrame({"age": [20, 21], "qx": [0.001, 0.002]})

        # Act
        result = gs.load_assumptions("test_table", df)

        # Assert
        assert len(result) == 2
        assert "age" in result.columns
```

**Performance Tests**:

```python
import pytest

class TestNewFeaturePerformance:
    @pytest.mark.benchmark(group="new_feature")
    def test_new_feature_performance(self, benchmark):
        """Benchmark new feature performance."""
        def operation_to_benchmark():
            # Your operation here
            return gs.some_operation(data)

        result = benchmark(operation_to_benchmark)

        # Assertions about results and performance
        assert len(result) > 0
        assert benchmark.stats.stats.mean < 1.0  # Max 1 second
```

### Test Markers

Use appropriate markers for your tests:

```python
# docs-skip
@pytest.mark.benchmark(group="operation_name")  # Performance tests
@pytest.mark.slow                               # Slow-running tests
@pytest.mark.skip(reason="Not implemented")     # Skip temporarily
```

## Debugging Tests

### Verbose Output

```bash
# Run with verbose output
uv run pytest -v

# Show all output (including print statements)
uv run pytest -s

# Stop on first failure
uv run pytest -x

# Run last failed tests only
uv run pytest --lf
```

### Performance Test Output

```bash
# Show benchmark statistics
uv run pytest -m benchmark --benchmark-only --benchmark-verbose

# Save benchmark results to file
uv run pytest -m benchmark --benchmark-json=results.json

# Compare benchmark results
uv run pytest -m benchmark --benchmark-compare=baseline.json
```

## Environment Setup

### Skip Performance Tests Permanently

If you want to skip performance tests for an entire development session:

```bash
# Set environment variable
export SKIP_PERFORMANCE_TESTS=true
uv run pytest  # Will skip all performance tests

# Or run once
SKIP_PERFORMANCE_TESTS=true uv run pytest
```

### Shell Aliases

Add these to your shell profile (`.zshrc`, `.bashrc`) for convenience:

```bash
# Fast development testing (no benchmarks)
alias test-fast="uv run pytest"

# Performance testing only  
alias test-perf="uv run pytest -m benchmark"

# All tests including performance
alias test-all="uv run pytest -m ''"

# Verbose testing for debugging
alias test-debug="uv run pytest -v -s"
```

## Continuous Integration

In CI environments, the test configuration automatically:

- Uses smaller datasets for performance tests
- Adjusts time limits for different machine capabilities
- Skips performance tests by default (same as local development)
- Can run performance tests explicitly when needed for benchmarking

To run performance tests in CI:

```bash
# In CI scripts
uv run pytest -m benchmark  # Explicit performance testing
```

## Troubleshooting

### Common Issues

**Tests running slowly**: Make sure you're not accidentally running performance tests:

```bash
# Check what tests are selected
uv run pytest --collect-only -q
```

**Performance tests failing**: Check if you're in a resource-constrained environment:

```bash
# Run with relaxed performance requirements
CI=true uv run pytest -m benchmark
```

**Import errors**: Make sure you're in the correct directory and have built the package:

```bash
cd bindings/python
uv run pytest
```

### Getting Help

- Check test output with `-v` for verbose information
- Use `--tb=short` for shorter tracebacks
- Run `uv run pytest --markers` to see all available test markers
- Check the test files in `tests/` for examples of similar functionality
# API

## `gaspatchio.frame.base.ActuarialFrame`

A lazy, chainable, and traceable DataFrame for actuarial modeling.

The ActuarialFrame provides a high-level API for common actuarial calculations and data manipulations, leveraging Polars LazyFrames for performance. It supports tracing of operations for optimization and introspection, and provides convenient accessors for specialized functionality (e.g., date, finance, excel operations).

Parameters:

| Name      | Type   | Description | Default                                                                                                                                                                                                                                                    |
| --------- | ------ | ----------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `data`    | \`dict | DataFrame   | LazyFrame                                                                                                                                                                                                                                                  |
| `mode`    | \`str  | None\`      | The operational mode: "run", "optimize", or "debug". - "run": Executes operations eagerly. - "optimize": Defers execution and builds a computation graph. - "debug": Provides more verbose output. Defaults to the global default mode (get_default_mode). |
| `verbose` | \`bool | None\`      | Enables or disables verbose logging. Defaults to the global default verbosity (get_default_verbose).                                                                                                                                                       |
| `threads` | \`int  | None\`      | Number of threads for parallel operations. Defaults to a system-dependent value or \_DEFAULT_THREADS.                                                                                                                                                      |

Attributes:

| Name      | Type                   | Description                                    |
| --------- | ---------------------- | ---------------------------------------------- |
| `date`    | `DateFrameAccessor`    | Accessor for date-related operations.          |
| `excel`   | `ExcelFrameAccessor`   | Accessor for Excel-like operations.            |
| `finance` | `FinanceFrameAccessor` | Accessor for financial calculations.           |
| `columns` | `list[str]`            | A list of column names in their current order. |

Examples:

**Initialization and Basic Operations**

```pycon
>>> from gaspatchio import ActuarialFrame
>>> data = {
...     "policy_id": [1, 1, 2, 2, 3],
...     "inception_date": [
...         "2020-01-01",
...         "2020-01-01",
...         "2021-05-10",
...         "2021-05-10",
...         "2022-02-20",
...     ],
...     "premium": [100, 150, 200, 50, 300],
...     "claims": [0, 50, 10, 0, 120],
... }
>>> af = ActuarialFrame(data)
>>> af["loss_ratio"] = af["claims"] / af["premium"]
>>> result = af.collect()
>>> print(result.head(3))
shape: (3, 5)
┌───────────┬────────────────┬─────────┬────────┬────────────┐
│ policy_id ┆ inception_date ┆ premium ┆ claims ┆ loss_ratio │
│ ---       ┆ ---            ┆ ---     ┆ ---    ┆ ---        │
│ i64       ┆ str            ┆ i64     ┆ i64    ┆ f64        │
╞═══════════╪════════════════╪═════════╪════════╪════════════╡
│ 1         ┆ 2020-01-01     ┆ 100     ┆ 0      ┆ 0.0        │
│ 1         ┆ 2020-01-01     ┆ 150     ┆ 50     ┆ 0.333333   │
│ 2         ┆ 2021-05-10     ┆ 200     ┆ 10     ┆ 0.05       │
└───────────┴────────────────┴─────────┴────────┴────────────┘
```

**Using `sum` over a group**

```pycon
>>> af = ActuarialFrame(data)
>>> af["total_premium_per_policy"] = af["premium"].sum().over("policy_id")
>>> result_with_sum = af.collect()
>>> print(result_with_sum)
shape: (5, 5)
┌───────────┬────────────────┬─────────┬────────┬──────────────────────────┐
│ policy_id ┆ inception_date ┆ premium ┆ claims ┆ total_premium_per_policy │
│ ---       ┆ ---            ┆ ---     ┆ ---    ┆ ---                      │
│ i64       ┆ str            ┆ i64     ┆ i64    ┆ i64                      │
╞═══════════╪════════════════╪═════════╪════════╪══════════════════════════╡
│ 1         ┆ 2020-01-01     ┆ 100     ┆ 0      ┆ 250                      │
│ 1         ┆ 2020-01-01     ┆ 150     ┆ 50     ┆ 250                      │
│ 2         ┆ 2021-05-10     ┆ 200     ┆ 10     ┆ 250                      │
│ 2         ┆ 2021-05-10     ┆ 50      ┆ 0      ┆ 250                      │
│ 3         ┆ 2022-02-20     ┆ 300     ┆ 120    ┆ 300                      │
└───────────┴────────────────┴─────────┴────────┴──────────────────────────┘
```

**Using an accessor (e.g., date accessor)**

Assume 'inception_date' needs to be parsed to a date type first. For simplicity, let's imagine it's already a date type for this example. (Actual parsing would use `af["inception_date"].str.to_date("%Y-%m-%d")` or similar)

```pycon
>>> # If 'inception_date' was a date type:
>>> # af["inception_year"] = af.date.year("inception_date")
>>> # af_with_year = af.collect()
>>> # print(af_with_year.select(["policy_id", "inception_year"]))
```

### `columns`

Return the names of the columns in the current order.

### `date`

Access date-related frame operations.

### `excel`

Access excel-related frame operations.

### `finance`

Access finance-related frame operations.

### `projection`

Access projection setup and the rollforward builder for this frame.

A declared property like its `date`/`finance`/`excel` siblings — not registry-`__getattr__` — so the editor surface and runtime agree (gh#104).

### `collect(*, engine='streaming')`

Execute and materialize the dataframe.

`collect` is the public escape hatch from the lazy `ActuarialFrame` graph to an eager :class:`polars.DataFrame`. Inside a model function the lazy form is usually what you want — Polars fuses the expressions and avoids intermediate materialisation. Reach for `collect()` when the calculation genuinely needs eager column arrays, most commonly when handing per-policy probabilities to a numpy RNG inside a stochastic scenario kernel.

Parameters:

| Name     | Type  | Description                                                                                                                                                                                                                                                 | Default       |
| -------- | ----- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------- |
| `engine` | `str` | Execution engine to use. Options: - "streaming" (default): Process data in batches for ~2x faster execution. Falls back to in-memory for unsupported operations. - "in-memory": Classic Polars in-memory execution. - "auto": Let Polars choose the engine. | `'streaming'` |

Returns:

| Type        | Description                                           |
| ----------- | ----------------------------------------------------- |
| `DataFrame` | Materialized DataFrame with all computations applied. |

Examples:

Inside a `for_each_scenario` stochastic model function:

```pycon
>>> import numpy as np
>>> import polars as pl
>>> def my_model(af, *, tables, drivers):
...     df = af.collect()  # materialise for numpy RNG access
...     rng = np.random.default_rng(drivers["rng_seed"])
...     deaths = rng.binomial(1, df["q_mort"].to_numpy())
...     return af.with_columns(pl.Series("died", deaths))
```

### `count()`

Count non-null values in each column.

Returns a single-row frame containing the count of non-null values for each column. Essential for data quality assessment, completeness checks, and exposure calculations in actuarial analysis.

When to use

- **Data Quality:** Assess completeness of critical fields like policy ID, sum assured, or premium to identify missing data issues.
- **Exposure Calculation:** Count policies, lives, or claims for exposure-based calculations in pricing and reserving.
- **Cohort Analysis:** Determine size of different risk groups, age bands, or product segments for credibility assessment.
- **Validation:** Verify record counts match expected values after data processing, joins, or filtering operations.

##### Returns

CountResult A frame with one row containing non-null counts for each column.

##### Examples

**Scalar Example: Data Completeness Check**

```python
from gaspatchio import ActuarialFrame

data = {
    "policy_id": ["P001", "P002", "P003", "P004", None],
    "age": [25, 45, None, 35, 52],
    "sum_assured": [100000, 500000, 250000, None, 300000],
    "status": ["Active", "Active", "Lapsed", "Active", "Active"],
}
af = ActuarialFrame(data)
counts = af.count()
print(counts)
print("Complete policies:", counts["policy_id"])
print("Complete ages:", counts["age"])
print("Data completeness %:", counts["age"] / 5 * 100)
```

```text
shape: (1, 4)
┌───────────┬─────┬─────────────┬────────┐
│ policy_id ┆ age ┆ sum_assured ┆ status │
│ ---       ┆ --- ┆ ---         ┆ ---    │
│ u32       ┆ u32 ┆ u32         ┆ u32    │
╞═══════════╪═════╪═════════════╪════════╡
│ 4         ┆ 4   ┆ 4           ┆ 5      │
└───────────┴─────┴─────────────┴────────┘
Complete policies: 4
Complete ages: 4
Data completeness %: 80.0
```

**Vector Example: Monthly Activity Counts**

```python
from gaspatchio import ActuarialFrame

data = {
    "month": ["Jan", "Feb"],
    "daily_claims": [
        [5, 3, 0, 4, None, 2, 1, 0, 3, None, 4, 2, 0, 1, 5],
        [2, None, 3, 1, 0, 4, None, 2, 0, 3, 1, None, 4, 2, 0]
    ],
    "daily_lapses": [
        [1, 0, 0, 2, 1, 0, 0, 1, 0, 0, 1, 0, 2, 0, 1],
        [0, 1, 0, 0, 2, 0, 1, 0, 1, 0, 0, 1, 0, 2, 0]
    ]
}
af = ActuarialFrame(data)

# Count valid daily observations
counts = af.count()
print(counts)
```

```text
shape: (1, 3)
┌───────┬──────────────┬──────────────┐
│ month ┆ daily_claims ┆ daily_lapses │
│ ---   ┆ ---          ┆ ---          │
│ u32   ┆ u32          ┆ u32          │
╞═══════╪══════════════╪══════════════╡
│ 2     ┆ 2            ┆ 2            │
└───────┴──────────────┴──────────────┘
```

### `drop(*columns)`

Remove columns from the frame.

Drops intermediate or debug columns that are no longer needed. Commonly used at the end of a model to clean up temporary calculations before writing results.

When to use

- **Cleanup:** Remove intermediate calculation columns (flags, temporary rates) before writing final results to parquet.
- **Memory:** Drop large columns that are no longer needed to reduce peak memory during collection.

##### Parameters

\*columns : str Column names to drop.

##### Returns

ActuarialFrame Frame without the specified columns.

##### Examples

**Remove temporary columns before output**

```python
from gaspatchio import ActuarialFrame

af = ActuarialFrame(
    {
        "policy_id": ["P001"],
        "premium": [1200],
        "sum_assured": [100000],
        "temp_debug": [999],
    }
)

af = af.drop("temp_debug")

print(af.collect())
```

```text
shape: (1, 3)
┌───────────┬─────────┬─────────────┐
│ policy_id ┆ premium ┆ sum_assured │
│ ---       ┆ ---     ┆ ---         │
│ str       ┆ i64     ┆ i64         │
╞═══════════╪═════════╪═════════════╡
│ P001      ┆ 1200    ┆ 100000      │
└───────────┴─────────┴─────────────┘
```

### `fill_series(column, start=0, increment=1)`

Apply fill_series using the core function.

### `filter(predicate)`

Filter rows by a boolean expression.

Removes policies that don't match the condition. Commonly used to exclude lapsed, matured, or otherwise inactive policies before running a projection.

When to use

- **In-Force Selection:** Filter to active policies (`status == "IF"`) before projection to exclude lapsed, surrendered, or matured business.
- **Cohort Analysis:** Isolate a subset of policies by product, age band, or underwriting class for targeted analysis.

##### Parameters

predicate : pl.Expr Boolean expression to filter by.

##### Returns

ActuarialFrame Frame with only matching rows.

##### Examples

**Filter to in-force policies only**

```python
import polars as pl
from gaspatchio import ActuarialFrame

af = ActuarialFrame(
    {
        "policy_id": ["P001", "P002", "P003"],
        "status": ["IF", "LAPSED", "IF"],
        "premium": [1200, 800, 1500],
    }
)

af = af.filter(pl.col("status") == "IF")

print(af.collect())
```

```text
shape: (2, 3)
┌───────────┬────────┬─────────┐
│ policy_id ┆ status ┆ premium │
│ ---       ┆ ---    ┆ ---     │
│ str       ┆ str    ┆ i64     │
╞═══════════╪════════╪═════════╡
│ P001      ┆ IF     ┆ 1200    │
│ P003      ┆ IF     ┆ 1500    │
└───────────┴────────┴─────────┘
```

### `get_column_order()`

Return the tracked order of columns.

### `join(other, on=None, left_on=None, right_on=None, how='left', maintain_order=None)`

Join with another DataFrame without leaving the ActuarialFrame API.

Enriches model points with assumption parameters, product lookups, expense tables, or any static data that should be attached per policy. This replaces the pattern of calling `.collect()` + raw Polars join

- re-wrapping with `ActuarialFrame()`.

When to use

- **Expense Parameters:** Attach product-level expense loadings, commission rates, or overhead allocations to each policy before projection.
- **Product Configuration:** Join product parameter tables (riders, benefit features, premium patterns) onto model points by product code.
- **Cohort Enrichment:** Add portfolio-level or cohort-level attributes (risk class, distribution channel) from external reference data.

##### Parameters

other : pl.DataFrame | pl.LazyFrame The right-side table to join against. on : str | list[str] | None Column name(s) to join on (when both sides use the same name). left_on : str | list[str] | None Column name(s) on the left (this frame). right_on : str | list[str] | None Column name(s) on the right (other frame). how : str, default "left" Join type: "left", "inner", "outer", "cross". maintain_order : str | None, default None Row-order guarantee, passed through to Polars: "left" preserves this frame's row order, "right" the other side's, "left_right" / "right_left" both (primary side first). The default leaves order unspecified — under lazy/streaming execution the engine may reorder rows, so pass "left" when downstream logic depends on model-point order rather than sorting afterwards.

##### Returns

ActuarialFrame Frame with columns from both sides.

##### Examples

**Attach expense parameters by product code**

```python
import polars as pl
from gaspatchio import ActuarialFrame

expense_params = pl.DataFrame(
    {
        "product_code": ["TERM", "WL", "UL"],
        "expense_pct": [0.05, 0.08, 0.10],
    }
)

af = ActuarialFrame(
    {
        "policy_id": ["P001", "P002", "P003"],
        "product_code": ["TERM", "WL", "TERM"],
        "sum_assured": [100000, 200000, 150000],
    }
)

af = af.join(expense_params, on="product_code")

print(af.collect())
```

```text
shape: (3, 4)
┌───────────┬──────────────┬─────────────┬─────────────┐
│ policy_id ┆ product_code ┆ sum_assured ┆ expense_pct │
│ ---       ┆ ---          ┆ ---         ┆ ---         │
│ str       ┆ str          ┆ i64         ┆ f64         │
╞═══════════╪══════════════╪═════════════╪═════════════╡
│ P001      ┆ TERM         ┆ 100000      ┆ 0.05        │
│ P002      ┆ WL           ┆ 200000      ┆ 0.08        │
│ P003      ┆ TERM         ┆ 150000      ┆ 0.05        │
└───────────┴──────────────┴─────────────┴─────────────┘
```

### `max()`

Calculate maximum values across all numeric columns.

Returns a single-row frame containing the maximum value for each column. Essential for identifying outliers, validating data ranges, and determining upper bounds in actuarial calculations.

When to use

- **Data Validation:** Identify outliers in premium amounts, sum assured, or claim values that may require investigation.
- **Experience Analysis:** Find maximum claim amounts, policy sizes, or ages in a portfolio for risk assessment.
- **Regulatory Reporting:** Determine maximum exposure amounts for solvency calculations and stress testing.
- **Pricing Boundaries:** Identify upper limits for age bands, benefit amounts, or policy terms in product design.

##### Returns

MaxResult A frame with one row containing maximum values for each column.

##### Examples

**Scalar Example: Portfolio Maximum Values**

```python
from gaspatchio import ActuarialFrame

data = {
    "policy_id": ["P001", "P002", "P003", "P004"],
    "age": [25, 45, 67, 35],
    "sum_assured": [100000, 500000, 250000, 1000000],
    "annual_premium": [1200, 6000, 8500, 15000],
}
af = ActuarialFrame(data)
max_values = af.max()
print(max_values)
print("Max age:", max_values["age"][0])
print("Max sum assured:", max_values["sum_assured"][0])
```

```text
shape: (1, 4)
┌───────────┬─────┬─────────────┬────────────────┐
│ policy_id ┆ age ┆ sum_assured ┆ annual_premium │
│ ---       ┆ --- ┆ ---         ┆ ---            │
│ str       ┆ i64 ┆ i64         ┆ i64            │
╞═══════════╪═════╪═════════════╪════════════════╡
│ P004      ┆ 67  ┆ 1000000     ┆ 15000          │
└───────────┴─────┴─────────────┴────────────────┘
Max age: 67
Max sum assured: 1000000
```

**Vector Example: Maximum Monthly Claims**

```python
from gaspatchio import ActuarialFrame

data = {
    "policy_id": ["P001", "P002"],
    "policy_year": [1, 2],
    "monthly_claims": [
        [0, 500, 0, 1200, 0, 0, 800, 0, 0, 0, 0, 2500],
        [0, 0, 3000, 0, 0, 1500, 0, 0, 0, 4000, 0, 0]
    ],
    "monthly_premiums": [
        [1000, 1000, 1000, 1000, 1000, 1000, 1000, 1000, 1000, 1000, 1000, 1000],
        [1500, 1500, 1500, 1500, 1500, 1500, 1500, 1500, 1500, 1500, 1500, 1500]
    ]
}
af = ActuarialFrame(data)

# Get maximum values to understand worst-case scenarios
max_values = af.max()
print(max_values)
print("Max policy year:", max_values["policy_year"][0])
```

```text
shape: (1, 4)
┌───────────┬─────────────┬─────────────────────────────────────┬─────────────────────────────────────┐
│ policy_id ┆ policy_year ┆ monthly_claims                      ┆ monthly_premiums                    │
│ ---       ┆ ---         ┆ ---                                 ┆ ---                                 │
│ str       ┆ i64         ┆ list[i64]                           ┆ list[i64]                           │
╞═══════════╪═════════════╪═════════════════════════════════════╪═════════════════════════════════════╡
│ P002      ┆ 2           ┆ [0, 500, 3000, 1200, … 4000, 0, 0]  ┆ [1500, 1500, 1500, 1500, … 1500]    │
└───────────┴─────────────┴─────────────────────────────────────┴─────────────────────────────────────┘
Max policy year: 2
```

### `mean()`

Calculate mean values across all numeric columns.

Returns a single-row frame containing the mean value for each numeric column. Essential for portfolio analysis, experience studies, and establishing benchmarks in actuarial calculations.

When to use

- **Experience Analysis:** Calculate average claim amounts, policy sizes, or premium levels for portfolio segmentation and pricing.
- **Trend Analysis:** Determine average lapse rates, mortality rates, or expense ratios over observation periods.
- **Benchmarking:** Establish portfolio averages for age, sum assured, or duration to compare against industry standards.
- **Reserve Calculations:** Compute average policy values, benefit amounts, or reserve factors for grouped calculations.

##### Returns

MeanResult A frame with one row containing mean values for numeric columns.

##### Examples

**Scalar Example: Portfolio Averages**

```python
from gaspatchio import ActuarialFrame

data = {
    "policy_id": ["P001", "P002", "P003", "P004"],
    "age": [25, 45, 67, 35],
    "sum_assured": [100000, 500000, 250000, 1000000],
    "annual_premium": [1200, 6000, 8500, 15000],
}
af = ActuarialFrame(data)
mean_values = af.mean()
print(mean_values)
print("Average age:", mean_values["age"])
print("Average sum assured:", mean_values["sum_assured"])
```

```text
shape: (1, 3)
┌──────┬──────────────┬─────────────────┐
│ age  ┆ sum_assured  ┆ annual_premium  │
│ ---  ┆ ---          ┆ ---             │
│ f64  ┆ f64          ┆ f64             │
╞══════╪══════════════╪═════════════════╡
│ 43.0 ┆ 462500.0     ┆ 7425.0          │
└──────┴──────────────┴─────────────────┘
Average age: 43.0
Average sum assured: 462500.0
```

**Vector Example: Average Monthly Experience**

```python
from gaspatchio import ActuarialFrame

data = {
    "policy_id": ["P001", "P002"],
    "policy_year": [1, 2],
    "monthly_claims": [
        [0, 500, 0, 1200, 0, 0, 800, 0, 0, 0, 0, 2500],
        [0, 0, 3000, 0, 0, 1500, 0, 0, 0, 4000, 0, 0]
    ],
    "monthly_lapses": [
        [2, 1, 3, 0, 1, 2, 1, 0, 1, 0, 2, 1],
        [1, 0, 2, 1, 0, 1, 0, 1, 0, 2, 1, 0]
    ]
}
af = ActuarialFrame(data)

# Get average monthly experience
mean_values = af.mean()
print(mean_values)
```

```text
shape: (1, 4)
┌─────────────┬───────────────────────────────┬──────────────────────────────┐
│ policy_year ┆ monthly_claims                ┆ monthly_lapses               │
│ ---         ┆ ---                           ┆ ---                          │
│ f64         ┆ list[f64]                     ┆ list[f64]                    │
╞═════════════╪═══════════════════════════════╪══════════════════════════════╡
│ 1.5         ┆ [0.0, 250.0, 1500.0, … 0.0]   ┆ [1.5, 0.5, 2.5, … 0.5]       │
└─────────────┴───────────────────────────────┴──────────────────────────────┘
```

### `median()`

Calculate median values across all numeric columns.

Returns a single-row frame containing the median value for each numeric column. Useful for robust central tendency measures that are less affected by outliers in actuarial data.

When to use

- **Robust Analysis:** Use median instead of mean when data contains outliers, such as large claims or extreme ages in the portfolio.
- **Income Analysis:** Analyze median policyholder income or premium levels for market segmentation and product design.
- **Experience Studies:** Calculate median time to claim, policy duration, or age at lapse for more representative measures.
- **Pricing Benchmarks:** Determine median rates or factors when comparing across competitors or market segments.

##### Returns

MedianResult A frame with one row containing median values for numeric columns.

##### Examples

**Scalar Example: Median Policy Metrics**

```python
from gaspatchio import ActuarialFrame

data = {
    "policy_id": ["P001", "P002", "P003", "P004", "P005"],
    "duration_years": [1, 3, 5, 7, 15],
    "annual_premium": [1200, 3500, 2800, 4200, 12000],
    "age": [25, 35, 42, 38, 65],
}
af = ActuarialFrame(data)
median_values = af.median()
print(median_values)
print("Median duration:", median_values["duration_years"])
print("Median premium:", median_values["annual_premium"])
```

```text
shape: (1, 3)
┌────────────────┬────────────────┬──────┐
│ duration_years ┆ annual_premium ┆ age  │
│ ---            ┆ ---            ┆ ---  │
│ f64            ┆ f64            ┆ f64  │
╞════════════════╪════════════════╪══════╡
│ 5.0            ┆ 3500.0         ┆ 38.0 │
└────────────────┴────────────────┴──────┘
Median duration: 5.0
Median premium: 3500.0
```

**Vector Example: Median Monthly Performance**

```python
from gaspatchio import ActuarialFrame

data = {
    "agent": ["A001", "A002"],
    "monthly_sales": [
        [3, 5, 2, 8, 4, 6, 3, 7, 5, 4, 6, 9],
        [12, 15, 10, 18, 14, 16, 11, 20, 13, 17, 15, 22]
    ],
    "monthly_commission": [
        [450, 750, 300, 1200, 600, 900, 450, 1050, 750, 600, 900, 1350],
        [1800, 2250, 1500, 2700, 2100, 2400, 1650, 3000, 1950, 2550, 2250, 3300]
    ]
}
af = ActuarialFrame(data)

# Calculate median for typical performance assessment
median_values = af.median()
print(median_values)
print("Agent A001 median sales:", median_values["monthly_sales"][0])
print("Agent A002 median sales:", median_values["monthly_sales"][1])
```

```text
shape: (1, 3)
┌────────────┬────────────────────┬──────────────────────┐
│ agent      ┆ monthly_sales      ┆ monthly_commission   │
│ ---        ┆ ---                ┆ ---                  │
│ str        ┆ list[f64]          ┆ list[f64]            │
╞════════════╪════════════════════╪══════════════════════╡
│ null       ┆ [5.0, 15.0]        ┆ [750.0, 2250.0]      │
└────────────┴────────────────────┴──────────────────────┘
Agent A001 median sales: 5.0
Agent A002 median sales: 15.0
```

### `min()`

Calculate minimum values across all numeric columns.

Returns a single-row frame containing the minimum value for each column. Essential for identifying baseline values, detecting anomalies, and establishing lower bounds in actuarial calculations.

When to use

- **Data Quality Checks:** Identify potential data errors like negative ages, zero premiums, or missing values coded as extreme minimums.
- **Portfolio Analysis:** Find minimum entry ages, smallest policy sizes, or lowest premium amounts for market segmentation.
- **Risk Assessment:** Determine minimum coverage levels, deductibles, or retention limits in reinsurance analysis.
- **Product Design:** Establish minimum benefit guarantees, surrender values, or contribution limits for new products.

##### Returns

MinResult A frame with one row containing minimum values for each column.

##### Examples

**Scalar Example: Portfolio Minimum Values**

```python
from gaspatchio import ActuarialFrame

data = {
    "policy_id": ["P001", "P002", "P003", "P004"],
    "age": [25, 45, 67, 35],
    "sum_assured": [100000, 500000, 250000, 1000000],
    "annual_premium": [1200, 6000, 8500, 15000],
}
af = ActuarialFrame(data)
min_values = af.min()
print(min_values)
print("Min age:", min_values["age"])
print("Min sum assured:", min_values["sum_assured"])
```

```text
shape: (1, 4)
┌───────────┬─────┬─────────────┬────────────────┐
│ policy_id ┆ age ┆ sum_assured ┆ annual_premium │
│ ---       ┆ --- ┆ ---         ┆ ---            │
│ str       ┆ i64 ┆ i64         ┆ i64            │
╞═══════════╪═════╪═════════════╪════════════════╡
│ P001      ┆ 25  ┆ 100000      ┆ 1200           │
└───────────┴─────┴─────────────┴────────────────┘
Min age: 25
Min sum assured: 100000
```

**Vector Example: Minimum Monthly Claims**

```python
from gaspatchio import ActuarialFrame

data = {
    "policy_id": ["P001", "P002"],
    "policy_year": [1, 2],
    "monthly_claims": [
        [0, 500, 0, 1200, 0, 0, 800, 0, 0, 0, 0, 2500],
        [0, 0, 3000, 0, 0, 1500, 0, 0, 0, 4000, 0, 0]
    ],
    "monthly_retention": [
        [1000, 1000, 1000, 1000, 1000, 1000, 1000, 1000, 1000, 1000, 1000, 1000],
        [500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500]
    ]
}
af = ActuarialFrame(data)

# Get minimum values to understand retention levels
min_values = af.min()
print(min_values)
print("Min retention level:", min_values["monthly_retention"])
```

```text
shape: (1, 4)
┌───────────┬─────────────┬─────────────────────────────────────┬─────────────────────────────────────┐
│ policy_id ┆ policy_year ┆ monthly_claims                      ┆ monthly_retention                   │
│ ---       ┆ ---         ┆ ---                                 ┆ ---                                 │
│ str       ┆ i64         ┆ list[i64]                           ┆ list[i64]                           │
╞═══════════╪═════════════╪═════════════════════════════════════╪═════════════════════════════════════╡
│ P001      ┆ 1           ┆ [0, 0, 0, 0, … 0, 0, 0]             ┆ [500, 500, 500, 500, … 500]         │
└───────────┴─────────────┴─────────────────────────────────────┴─────────────────────────────────────┘
Min retention level: [500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, 500]
```

### `pipe(func, *args, **kwargs)`

Apply a function that accepts and returns an ActuarialFrame.

### `product()`

Calculate the product of values in each numeric column.

Returns a single-row frame containing the product of all values for each numeric column. Useful for compound calculations, probability chains, and multiplicative factors in actuarial modeling.

When to use

- **Compound Interest:** Calculate accumulated values using multiple period growth factors or discount factors.
- **Probability Chains:** Multiply survival probabilities, persistency rates, or success rates across multiple periods.
- **Factor Application:** Apply multiple adjustment factors, loading factors, or credibility factors in sequence.
- **Index Calculations:** Compute cumulative index values from period-to-period change factors.

##### Returns

ProductResult A frame with one row containing products for numeric columns.

##### Examples

**Scalar Example: Survival Probability Chain**

```python
from gaspatchio import ActuarialFrame

data = {
    "year": [1, 2, 3, 4, 5],
    "annual_survival": [0.999, 0.998, 0.997, 0.995, 0.993],
    "annual_persistency": [0.95, 0.92, 0.90, 0.88, 0.85],
}
af = ActuarialFrame(data)
products = af.product()
print(products)
print("5-year survival probability:", round(products["annual_survival"], 6))
print("5-year persistency:", round(products["annual_persistency"], 4))
```

```text
shape: (1, 3)
┌──────┬─────────────────┬────────────────────┐
│ year ┆ annual_survival ┆ annual_persistency │
│ ---  ┆ ---             ┆ ---                │
│ i64  ┆ f64             ┆ f64                │
╞══════╪═════════════════╪════════════════════╡
│ 120  ┆ 0.982089        ┆ 0.59262            │
└──────┴─────────────────┴────────────────────┘
5-year survival probability: 0.982089
5-year persistency: 0.5926
```

**Vector Example: Discount Factor Chains**

```python
from gaspatchio import ActuarialFrame

data = {
    "scenario": ["Base", "Stressed"],
    "monthly_discount": [
        [0.9992, 0.9992, 0.9992, 0.9992, 0.9992, 0.9992],
        [0.9990, 0.9990, 0.9990, 0.9990, 0.9990, 0.9990]
    ],
    "monthly_survival": [
        [0.9999, 0.9999, 0.9999, 0.9999, 0.9999, 0.9999],
        [0.9998, 0.9998, 0.9998, 0.9998, 0.9998, 0.9998]
    ]
}
af = ActuarialFrame(data)

# Calculate cumulative factors
products = af.product()
print(products)
```

```text
shape: (1, 3)
┌──────────┬──────────────────┬──────────────────┐
│ scenario ┆ monthly_discount ┆ monthly_survival │
│ ---      ┆ ---              ┆ ---              │
│ str      ┆ list[f64]        ┆ list[f64]        │
╞══════════╪══════════════════╪══════════════════╡
│ null     ┆ [0.9952, 0.9940] ┆ [0.9994, 0.9988] │
└──────────┴──────────────────┴──────────────────┘
```

### `profile()`

Execute and materialize the dataframe with profiling, returning (result_df, profile_info).

### `quantile(quantile, interpolation='nearest')`

Calculate quantile values across all numeric columns.

Returns a single-row frame containing the specified quantile for each numeric column. Essential for risk assessment, percentile-based analysis, and regulatory reporting in actuarial applications.

When to use

- **Risk Assessment:** Calculate VaR (Value at Risk) at different confidence levels (e.g., 95th, 99th percentile) for solvency calculations.
- **Experience Analysis:** Determine percentile thresholds for large claims, high-risk ages, or outlier detection in portfolios.
- **Pricing Segmentation:** Identify quantile boundaries for premium bands, risk tiers, or underwriting categories.
- **Regulatory Reporting:** Calculate required percentiles for stress testing, capital requirements, or reserve adequacy testing.

##### Parameters

quantile : float Quantile value between 0 and 1 (e.g., 0.5 for median, 0.95 for 95th percentile). interpolation : str, default "nearest" Interpolation method: "nearest", "higher", "lower", "midpoint", or "linear".

##### Returns

QuantileResult A frame with one row containing quantile values for numeric columns.

##### Examples

**Scalar Example: Claims Distribution Analysis**

```python
from gaspatchio import ActuarialFrame

data = {
    "claim_id": list(range(1, 101)),
    "claim_amount": [
        1000,
        1500,
        2000,
        2500,
        3000,
        3500,
        4000,
        5000,
        6000,
        7500,
        8000,
        9000,
        10000,
        12000,
        15000,
        18000,
        20000,
        25000,
        30000,
        35000,
        40000,
        45000,
        50000,
        60000,
        75000,
        85000,
        95000,
        100000,
        120000,
        150000,
    ]
    + [2000] * 70,
    "processing_days": list(range(5, 35)) + list(range(10, 80)),
}
af = ActuarialFrame(data)

# Calculate key percentiles
p90 = af.quantile(0.90)
p95 = af.quantile(0.95)
p99 = af.quantile(0.99)

print("90th percentile:")
print(p90)
print("\\nClaim amount 90th percentile:", p90["claim_amount"])
print("Claim amount 95th percentile:", p95["claim_amount"])
print("Claim amount 99th percentile:", p99["claim_amount"])
```

```text
90th percentile:
shape: (1, 3)
┌──────────┬──────────────┬─────────────────┐
│ claim_id ┆ claim_amount ┆ processing_days │
│ ---      ┆ ---          ┆ ---             │
│ f64      ┆ f64          ┆ f64             │
╞══════════╪══════════════╪═════════════════╡
│ 90.0     ┆ 85000.0      ┆ 71.0            │
└──────────┴──────────────┴─────────────────┘

Claim amount 90th percentile: 85000.0
Claim amount 95th percentile: 100000.0
Claim amount 99th percentile: 150000.0
```

**Vector Example: Portfolio Risk Percentiles**

```python
from gaspatchio import ActuarialFrame

data = {
    "product": ["Term Life", "Whole Life"],
    "claim_amounts": [
        [10000, 15000, 20000, 25000, 30000, 35000, 40000, 50000, 75000, 100000,
         150000, 200000, 250000, 300000, 500000, 750000, 1000000, 1500000, 2000000, 3000000],
        [50000, 75000, 100000, 125000, 150000, 175000, 200000, 250000, 300000, 400000,
         500000, 600000, 750000, 900000, 1000000, 1250000, 1500000, 2000000, 2500000, 5000000]
    ]
}
af = ActuarialFrame(data)

# Calculate 95th percentile for risk assessment
var_95 = af.quantile(0.95)
print("95% VaR by product:")
print(var_95)
```

```text
95% VaR by product:
shape: (1, 2)
┌────────────┬──────────────────────────────────┐
│ product    ┆ claim_amounts                    │
│ ---        ┆ ---                              │
│ str        ┆ list[f64]                        │
╞════════════╪══════════════════════════════════╡
│ null       ┆ [2000000.0, 2500000.0]           │
└────────────┴──────────────────────────────────┘
```

### `rename(mapping)`

Rename columns to snake_case or actuarial conventions.

Converts raw data column names (often from Excel or vendor systems) to the snake_case convention used throughout Gaspatchio models. Run this in Phase 1 (setup) before creating the projection timeline.

When to use

- **Data Ingestion:** Convert Excel-style column names (`"Issue Age"`, `"Sum Assured"`) to snake_case for use as ActuarialFrame attributes.
- **Vendor Data:** Standardise column names from different admin systems or data providers before building assumptions.

##### Parameters

mapping : dict[str, str] Mapping of old name to new name.

##### Returns

ActuarialFrame Frame with renamed columns.

##### Examples

**Rename Excel-style columns to snake_case**

```python
from gaspatchio import ActuarialFrame

af = ActuarialFrame(
    {
        "Policy Number": ["P001", "P002"],
        "Issue Age": [30, 45],
        "Sum Assured": [100000, 200000],
    }
)

af = af.rename(
    {
        "Policy Number": "policy_id",
        "Issue Age": "issue_age",
        "Sum Assured": "sum_assured",
    }
)

print(af.collect())
```

```text
shape: (2, 3)
┌───────────┬───────────┬─────────────┐
│ policy_id ┆ issue_age ┆ sum_assured │
│ ---       ┆ ---       ┆ ---         │
│ str       ┆ i64       ┆ i64         │
╞═══════════╪═══════════╪═════════════╡
│ P001      ┆ 30        ┆ 100000      │
│ P002      ┆ 45        ┆ 200000      │
└───────────┴───────────┴─────────────┘
```

### `select(*exprs, **named_exprs)`

Select columns from the DataFrame.

Accepts positional expressions (column names, proxies, or expressions) and keyword arguments for renamed/new expressions.

Parameters:

| Name            | Type             | Description                                       | Default |
| --------------- | ---------------- | ------------------------------------------------- | ------- |
| `*exprs`        | `IntoExprColumn` | Columns or expressions to select.                 | `()`    |
| `**named_exprs` | `IntoExprColumn` | Expressions to select with specific output names. | `{}`    |

Returns:

| Type   | Description                  |
| ------ | ---------------------------- |
| `Self` | The modified ActuarialFrame. |

### `show_query_plan(enabled=True)`

Enable or disable query plan logging (basic implementation).

### `sort(by, *, descending=False)`

Sort rows by one or more columns.

Orders policies by a key column before projection or output. Useful for deterministic output ordering in reconciliation.

When to use

- **Reconciliation:** Sort by policy ID before comparing against a reference model to ensure row-by-row alignment.
- **Reporting:** Order output by issue age, premium, or product code for presentation.

##### Parameters

by : str | list[str] Column name(s) to sort by. descending : bool, default False Sort in descending order.

##### Returns

ActuarialFrame Sorted frame.

##### Examples

**Sort by issue age for reporting**

```python
from gaspatchio import ActuarialFrame

af = ActuarialFrame(
    {
        "policy_id": ["P003", "P001", "P002"],
        "issue_age": [55, 30, 45],
        "premium": [1500, 1200, 800],
    }
)

af = af.sort("issue_age")

print(af.collect())
```

```text
shape: (3, 3)
┌───────────┬───────────┬─────────┐
│ policy_id ┆ issue_age ┆ premium │
│ ---       ┆ ---       ┆ ---     │
│ str       ┆ i64       ┆ i64     │
╞═══════════╪═══════════╪═════════╡
│ P001      ┆ 30        ┆ 1200    │
│ P002      ┆ 45        ┆ 800     │
│ P003      ┆ 55        ┆ 1500    │
└───────────┴───────────┴─────────┘
```

### `std(ddof=1)`

Calculate standard deviation across all numeric columns.

Returns a single-row frame containing the standard deviation for each numeric column. Essential for risk assessment, volatility analysis, and confidence interval calculations in actuarial modeling.

When to use

- **Risk Assessment:** Measure volatility in claim amounts, premium variations, or mortality experience for pricing and reserving.
- **Experience Monitoring:** Quantify variability in lapse rates, expense ratios, or benefit utilization for assumption setting.
- **Confidence Intervals:** Calculate standard errors for mortality estimates, reserve factors, or pricing assumptions.
- **Portfolio Analysis:** Assess homogeneity of risk groups by comparing standard deviations across segments.

##### Parameters

ddof : int, default 1 Delta degrees of freedom. The divisor is N - ddof.

##### Returns

StdResult A frame with one row containing standard deviations for numeric columns.

##### Examples

**Scalar Example: Premium Volatility Analysis**

```python
from gaspatchio import ActuarialFrame

data = {
    "policy_id": ["P001", "P002", "P003", "P004", "P005"],
    "age_band": ["25-35", "25-35", "36-45", "36-45", "46-55"],
    "annual_premium": [1200, 1350, 3500, 3200, 8500],
    "sum_assured": [100000, 150000, 350000, 300000, 500000],
}
af = ActuarialFrame(data)
std_values = af.std()
print(std_values)
print("Premium volatility:", std_values["annual_premium"])
```

```text
shape: (1, 2)
┌──────────────────┬─────────────┐
│ annual_premium   ┆ sum_assured │
│ ---              ┆ ---         │
│ f64              ┆ f64         │
╞══════════════════╪═════════════╡
│ 2913.8           ┆ 158113.9    │
└──────────────────┴─────────────┘
Premium volatility: 2913.8
```

**Vector Example: Monthly Claims Volatility**

```python
from gaspatchio import ActuarialFrame

data = {
    "product": ["Term Life", "Whole Life"],
    "monthly_claims": [
        [0, 1000, 500, 2000, 0, 3000, 1500, 0, 2500, 1000, 0, 4000],
        [5000, 6000, 4500, 7000, 5500, 8000, 6500, 5000, 7500, 6000, 9000, 10000]
    ],
    "monthly_premiums": [
        [50000, 50000, 52000, 51000, 50000, 49000, 50000, 51000, 50000, 50000, 51000, 50000],
        [120000, 125000, 122000, 128000, 124000, 130000, 126000, 123000, 127000, 125000, 129000, 132000]
    ]
}
af = ActuarialFrame(data)

# Calculate standard deviation for risk assessment
std_values = af.std()
print(std_values)
print("Term Life claims volatility:", round(std_values["monthly_claims"][0], 2))
print("Whole Life claims volatility:", round(std_values["monthly_claims"][1], 2))
```

```text
shape: (1, 3)
┌────────────┬──────────────────────────────┬───────────────────────────────┐
│ product    ┆ monthly_claims               ┆ monthly_premiums              │
│ ---        ┆ ---                          ┆ ---                           │
│ str        ┆ list[f64]                    ┆ list[f64]                     │
╞════════════╪══════════════════════════════╪═══════════════════════════════╡
│ null       ┆ [1443.38, 1443.38]           ┆ [831.66, 3207.14]             │
└────────────┴──────────────────────────────┴───────────────────────────────┘
Term Life claims volatility: 1443.38
Whole Life claims volatility: 1443.38
```

### `sum()`

Calculate sum totals across all numeric columns.

Returns a single-row frame containing the sum total for each numeric column. Critical for calculating portfolio totals, aggregate exposures, and overall metrics in actuarial reporting.

When to use

- **Portfolio Totals:** Calculate total sum assured, total premiums collected, or total claims paid for financial reporting.
- **Exposure Analysis:** Sum total lives covered, total benefits, or total risk amounts for reinsurance and capital calculations.
- **Revenue Reporting:** Aggregate premium income, fee revenue, or investment income across product lines or time periods.
- **Claims Analysis:** Total claim counts, amounts paid, or reserves across different claim types or cohorts.

##### Returns

SumResult A frame with one row containing sum totals for numeric columns.

##### Examples

**Scalar Example: Portfolio Totals**

```python
from gaspatchio import ActuarialFrame

data = {
    "product": ["Term", "Whole Life", "Universal", "Term", "Endowment"],
    "policies_inforce": [1250, 890, 445, 2100, 325],
    "annual_premium": [1500000, 3200000, 2100000, 2800000, 1900000],
    "sum_assured": [125000000, 89000000, 67000000, 315000000, 48000000],
}
af = ActuarialFrame(data)
sum_values = af.sum()
print(sum_values)
print("Total policies:", sum_values["policies_inforce"])
print("Total premium:", sum_values["annual_premium"])
print("Total exposure:", sum_values["sum_assured"])
```

```text
shape: (1, 3)
┌──────────────────┬────────────────┬─────────────┐
│ policies_inforce ┆ annual_premium ┆ sum_assured │
│ ---              ┆ ---            ┆ ---         │
│ i64              ┆ i64            ┆ i64         │
╞══════════════════╪════════════════╪═════════════╡
│ 5010             ┆ 11500000       ┆ 644000000   │
└──────────────────┴────────────────┴─────────────┘
Total policies: 5010
Total premium: 11500000
Total exposure: 644000000
```

**Vector Example: Monthly Totals**

```python
from gaspatchio import ActuarialFrame

data = {
    "branch": ["North", "South"],
    "monthly_new_business": [
        [120, 135, 110, 145, 130, 125, 140, 155, 135, 140, 130, 160],
        [95, 100, 90, 105, 110, 95, 100, 115, 105, 100, 95, 120]
    ],
    "monthly_premium": [
        [180000, 202500, 165000, 217500, 195000, 187500, 210000, 232500, 202500, 210000, 195000, 240000],
        [142500, 150000, 135000, 157500, 165000, 142500, 150000, 172500, 157500, 150000, 142500, 180000]
    ]
}
af = ActuarialFrame(data)

# Get total new business and premiums
sum_values = af.sum()
print(sum_values)
```

```text
shape: (1, 2)
┌───────────────────────────────────────┬───────────────────────────────────────┐
│ monthly_new_business                  ┆ monthly_premium                       │
│ ---                                   ┆ ---                                   │
│ list[i64]                             ┆ list[i64]                             │
╞═══════════════════════════════════════╪═══════════════════════════════════════╡
│ [215, 235, 200, 250, … 240, 225, 280] ┆ [322500, 352500, 300000, … 420000]    │
└───────────────────────────────────────┴───────────────────────────────────────┘
```

### `trace(func)`

Capture operations within a function call in optimize mode.

### `var(ddof=1)`

Calculate variance across all numeric columns.

Returns a single-row frame containing the variance for each numeric column. Used for risk metrics, ANOVA calculations, and statistical modeling in actuarial applications.

When to use

- **Risk Metrics:** Calculate variance in loss ratios, combined ratios, or expense ratios for enterprise risk management.
- **Statistical Testing:** Perform ANOVA on mortality rates, lapse rates, or claim frequencies across different cohorts.
- **Credibility Theory:** Calculate variance components for Bühlmann credibility factors in experience rating.
- **Asset-Liability Modeling:** Measure variance in investment returns, liability cash flows, or surplus positions.

##### Parameters

ddof : int, default 1 Delta degrees of freedom. The divisor is N - ddof.

##### Returns

VarResult A frame with one row containing variances for numeric columns.

##### Examples

**Scalar Example: Claims Variance Analysis**

```python
from gaspatchio import ActuarialFrame

data = {
    "month": [1, 2, 3, 4, 5, 6],
    "claims_count": [45, 52, 38, 61, 43, 55],
    "claims_amount": [125000, 145000, 95000, 185000, 120000, 165000],
}
af = ActuarialFrame(data)
var_values = af.var()
print(var_values)
print("Claims count variance:", var_values["claims_count"])
print("Claims amount variance:", var_values["claims_amount"])
```

```text
shape: (1, 3)
┌───────┬──────────────┬──────────────────┐
│ month ┆ claims_count ┆ claims_amount    │
│ ---   ┆ ---          ┆ ---              │
│ f64   ┆ f64          ┆ f64              │
╞═══════╪══════════════╪══════════════════╡
│ 3.5   ┆ 70.3         ┆ 1.091e9          │
└───────┴──────────────┴──────────────────┘
Claims count variance: 70.3
Claims amount variance: 1091000000.0
```

**Vector Example: Experience Variance Components**

```python
from gaspatchio import ActuarialFrame

data = {
    "region": ["North", "South"],
    "quarterly_lapse_rates": [
        [0.025, 0.028, 0.022, 0.026],
        [0.031, 0.029, 0.033, 0.030]
    ],
    "quarterly_mortality_rates": [
        [0.0010, 0.0011, 0.0009, 0.0010],
        [0.0012, 0.0013, 0.0011, 0.0014]
    ]
}
af = ActuarialFrame(data)

# Calculate variance for credibility analysis
var_values = af.var()
print(var_values)
print("North region lapse variance:", var_values["quarterly_lapse_rates"][0])
print("South region lapse variance:", var_values["quarterly_lapse_rates"][1])
```

```text
shape: (1, 3)
┌────────────┬────────────────────────┬──────────────────────────────┐
│ region     ┆ quarterly_lapse_rates  ┆ quarterly_mortality_rates    │
│ ---        ┆ ---                    ┆ ---                          │
│ str        ┆ list[f64]              ┆ list[f64]                    │
╞════════════╪════════════════════════╪══════════════════════════════╡
│ null       ┆ [0.000007, 0.000003]   ┆ [0.0000000067, 0.0000000167] │
└────────────┴────────────────────────┴──────────────────────────────┘
North region lapse variance: 0.000007
South region lapse variance: 0.000003
```

### `with_columns(*exprs)`

Add columns to the DataFrame.

# Assumptions API

## Table

## `gaspatchio.assumptions._api.Table`

Main assumption table class with dimension-based structure.

This class provides a clean API for creating assumption tables using composable dimension types and strategies, replacing the old monolithic load_assumptions() function.

### `dimensions`

Get dimension configuration (returns a copy).

Returns the dimension configuration used to structure this assumption table, providing access to dimension types, processing strategies, and validation rules for model analysis and debugging.

When to use

- **Model Analysis:** Inspect dimension configuration to understand table structure and lookup requirements.
- **Dynamic Lookups:** Build lookup calls programmatically based on available dimensions and their configurations.
- **Validation:** Check dimension compatibility when extending tables or building complex lookup expressions.

Returns:

| Type                   | Description                                             |
| ---------------------- | ------------------------------------------------------- |
| `dict[str, Dimension]` | dict\[str, Dimension\]: Copy of dimension configuration |

##### Examples:

```python
from gaspatchio.assumptions import Table
import polars as pl

data = pl.DataFrame({"age": [30, 35], "rate": [0.001, 0.002]})
table = Table("test", data, {"age": "age"}, "rate")

dims = table.dimensions
print(f"Dimensions: {list(dims.keys())}")
```

### `metadata`

Get metadata for this table.

Returns stored metadata for this assumption table including descriptions, data sources, validation status, and business context that was provided during table creation.

When to use

- **Documentation:** Access table metadata for automated documentation generation and model reporting.
- **Governance:** Retrieve data lineage, validation status, and review information for compliance reporting.
- **Model Management:** Check table metadata for version control, effective dates, and change management.

Returns:

| Type             | Description |
| ---------------- | ----------- |
| \`dict[str, Any] | None\`      |

##### Examples:

```python
from gaspatchio.assumptions import Table
import polars as pl

data = pl.DataFrame({"age": [30, 35], "rate": [0.001, 0.002]})
table = Table(
    "test", data, {"age": "age"}, "rate", metadata={"source": "2023 Study"}
)

meta = table.metadata
print(f"Source: {meta['source'] if meta else 'None'}")
```

### `name`

Get the table name supplied at construction.

Returns the stable identifier used by the registry, the typed-input audit trail (e.g. :class:`gaspatchio.MortalityTable.source_sha`), and any external consumer that needs a consistent reference for this table.

When to use

- **Audit trail**: Record the table's name alongside model results so reviewers can identify which assumption set produced each valuation.
- **Registry lookups**: Pass the name to other gaspatchio components (typed inputs, scenario configs) that need a consistent reference for this table across the model.
- **Reproducibility**: Pin the name into release manifests so a future rerun can confirm the same table was used.

Returns:

| Name  | Type  | Description                               |
| ----- | ----- | ----------------------------------------- |
| `str` | `str` | The name string passed to the constructor |

##### Examples:

```python
from gaspatchio.assumptions import Table
import polars as pl

data = pl.DataFrame({"age": [30, 35], "rate": [0.001, 0.002]})
table = Table("mortality_standard", data, {"age": "age"}, "rate")

print(table.name)
```

### `schema`

Get the analyzed schema of this table.

Returns comprehensive schema information about the assumption table including column types, value ranges, and structural metadata useful for validation, debugging, and documentation generation.

When to use

- **Data Validation:** Check table schema before model execution to ensure data types and ranges meet model requirements.
- **Debugging:** Inspect table structure when troubleshooting lookup failures or data quality issues.
- **Documentation:** Generate technical documentation showing table structure and data characteristics.

Returns:

| Name          | Type          | Description                             |
| ------------- | ------------- | --------------------------------------- |
| `TableSchema` | `TableSchema` | Analyzed schema with column information |

##### Examples:

```python
from gaspatchio.assumptions import Table
import polars as pl

data = pl.DataFrame({"age": [30, 35, 40], "rate": [0.001, 0.002, 0.004]})

table = Table("test", data, {"age": "age"}, "rate")
schema = table.schema
print(f"Columns: {len(schema.columns)}")
```

### `storage_mode`

Get the actual storage mode used by this table.

Returns the storage backend actually being used for lookups, which may differ from the requested mode when using "auto". This is useful for verifying that array storage was selected for dense tables.

When to use

- **Performance Verification:** Check if "auto" mode selected array storage (35x faster) or fell back to hash storage.
- **Debugging:** Verify storage mode when troubleshooting lookup performance issues.
- **Logging:** Record actual storage mode for model run diagnostics.

Returns:

| Name  | Type  | Description                                 |
| ----- | ----- | ------------------------------------------- |
| `str` | `str` | The actual storage mode - "hash" or "array" |

##### Examples:

```python
from gaspatchio.assumptions import Table
import polars as pl

# Dense table - should use array storage
data = pl.DataFrame(
    {
        "age": list(range(18, 101)),  # 83 ages
        "rate": [0.001 * (1 + a / 100) for a in range(18, 101)],
    }
)

table = Table(
    name="mortality_auto_test",
    source=data,
    dimensions={"age": "age"},
    value="rate",
    storage_mode="auto",  # Let Rust decide
)

print(f"Requested: auto, Actual: {table.storage_mode}")
# Output: Requested: auto, Actual: array
```

### `canonical_form()`

Deterministic JSON-encodable identity recipe for the audit chain.

Returns a dictionary that uniquely identifies this table's content and shape: name, sorted dimension keys, value column, and a row-order-independent SHA-256 of the underlying data. Two tables with the same content but loaded in different row orders produce the same canonical_form; two tables differing in any cell value produce different ones.

When to use

- **Audit chains:** Feed into `source_sha()` so a regulator can verify the table used in a SCR run by hash alone.
- **Reproducibility checks:** Confirm that a Table reloaded from disk matches the Table the run was authored against.
- **Change detection:** Compare canonical_form between versions to detect data drift without re-running models.

Returns:

| Type             | Description                                      |
| ---------------- | ------------------------------------------------ |
| `dict[str, Any]` | Dictionary with kind, name, sorted dimensions,   |
| `dict[str, Any]` | value_column, and a content content_sha over the |
| `dict[str, Any]` | row-sorted parquet bytes of the data.            |

##### Examples:

```python
import polars as pl
from gaspatchio.assumptions import Table

mortality = Table(
    name="mortality",
    source=pl.DataFrame({"age": [30, 31], "rate": [0.001, 0.0012]}),
    dimensions={"age": "age"},
    value="rate",
)

recipe = mortality.canonical_form()
print(sorted(recipe.keys()))
```

```text
['content_sha', 'dimensions', 'kind', 'name', 'value_column']
```

### `describe()`

Get a human-readable description of the table.

Returns a formatted string describing the table structure, including row count, column information, and dimension configuration. Useful for debugging, documentation, and model analysis.

When to use

- **Debugging:** Get quick overview of table structure when troubleshooting lookup issues or data problems.
- **Documentation:** Generate summary information for model documentation and technical specifications.
- **Model Analysis:** Review table characteristics during model development and validation processes.

Returns:

| Name  | Type  | Description                             |
| ----- | ----- | --------------------------------------- |
| `str` | `str` | Human-readable description of the table |

##### Examples:

```python
from gaspatchio.assumptions import Table
import polars as pl

data = pl.DataFrame({"age": [30, 35, 40], "rate": [0.001, 0.002, 0.004]})

table = Table("mortality", data, {"age": "age"}, "rate")
print(table.describe())
```

### `dimension_values(dimension)`

Get unique values for a specific dimension.

Returns a list of all unique values found in the specified dimension column of the assumption table. Useful for understanding the range of lookup keys available and for validation of lookup arguments.

When to use

- **Data Validation:** Check available dimension values before performing lookups to ensure valid lookup keys.
- **Model Analysis:** Examine the range of ages, durations, or product types covered by assumption tables.
- **Dynamic UI:** Build dropdown lists or selection interfaces showing available lookup values for assumption tables.

Parameters:

| Name        | Type  | Description                             | Default    |
| ----------- | ----- | --------------------------------------- | ---------- |
| `dimension` | `str` | Name of the dimension to get values for | *required* |

Returns:

| Type        | Description                                         |
| ----------- | --------------------------------------------------- |
| `list[Any]` | list\[Any\]: List of unique values in the dimension |

##### Examples:

```python
from gaspatchio.assumptions import Table
import polars as pl

data = pl.DataFrame(
    {"age": [30, 35, 40, 30, 35], "rate": [0.001, 0.002, 0.004, 0.001, 0.002]}
)

table = Table("test", data, {"age": "age"}, "rate")
ages = table.dimension_values("age")
print(f"Available ages: {sorted(ages)}")
```

### `extend(source, dimensions=None, validate=True)`

Extend table with additional data slices.

Appends additional data to an existing assumption table, allowing for incremental loading of assumption data from multiple sources or files. The new data undergoes the same dimension processing as the original table and becomes immediately available for lookups in model calculations.

When to use

- **Incremental Loading:** Add new assumption data slices from multiple files or data sources to build comprehensive tables.
- **Time-Based Updates:** Append new vintage data to existing assumption tables for model updates and refreshes.
- **Multi-Source Integration:** Combine assumption data from different systems, departments, or external providers.
- **Scenario Analysis:** Add alternative assumption sets to existing tables for stress testing and scenario modeling.

Parameters:

| Name         | Type                   | Description                       | Default                            |
| ------------ | ---------------------- | --------------------------------- | ---------------------------------- |
| `source`     | \`str                  | Path                              | DataFrame\`                        |
| `dimensions` | \`dict[str, Dimension] | None\`                            | Dimension overrides for this slice |
| `validate`   | `bool`                 | Whether to validate compatibility | `True`                             |

Returns:

| Type    | Description       |
| ------- | ----------------- |
| `Table` | Self for chaining |

##### Examples:

**Scalar Example: Extending Mortality Table**

```python
from gaspatchio.assumptions import Table
import polars as pl

# Create initial mortality table
initial_data = pl.DataFrame(
    {"age": [30, 35, 40], "mortality_rate": [0.001, 0.002, 0.004]}
)

mortality_table = Table(
    name="mortality_extended",
    source=initial_data,
    dimensions={"age": "age"},
    value="mortality_rate",
)

print(f"Initial rows: {len(mortality_table.to_dataframe())}")

# Extend with additional age bands
additional_data = pl.DataFrame(
    {"age": [45, 50, 55], "mortality_rate": [0.008, 0.015, 0.025]}
)

mortality_table.extend(source=additional_data)
print(f"After extension: {len(mortality_table.to_dataframe())}")
print(mortality_table.to_dataframe().sort("age"))
```

```text
Initial rows: 3
After extension: 6
shape: (6, 2)
┌──────┬────────────────┐
│ age  ┆ mortality_rate │
│ ---  ┆ ---            │
│ f64  ┆ f64            │
╞══════╪════════════════╡
│ 30.0 ┆ 0.001          │
│ 35.0 ┆ 0.002          │
│ 40.0 ┆ 0.004          │
│ 45.0 ┆ 0.008          │
│ 50.0 ┆ 0.015          │
│ 55.0 ┆ 0.025          │
└──────┴────────────────┘
```

**Vector Example: Multi-File Lapse Rate Integration**

```python
from gaspatchio.assumptions import Table
import polars as pl

# Create base mortality table
base_data = pl.DataFrame({
    "age": [30, 35, 40],
    "rate": [0.001, 0.002, 0.004]
})

table = Table(
    name="mortality_extended",
    source=base_data,
    dimensions={"age": "age"},
    value="rate"
)

print("Initial rows:", len(table.to_dataframe()))

# Extend with additional ages
additional_data = pl.DataFrame({
    "age": [45, 50],
    "rate": [0.008, 0.015]
})

table.extend(source=additional_data)
print("After extension:", len(table.to_dataframe()))
```

```text
Initial rows: 3
After extension: 5
```

### `from_scenario_files(scenario_files, scenario_column, dimensions, value, name=None, validate=True, metadata=None)`

Create a Table by concatenating per-scenario assumption files.

Loads each file, adds scenario_column with the scenario ID, concatenates all into a single DataFrame, and creates a Table with scenario_column as an additional dimension.

This is useful when assumptions are stored as separate files per scenario (e.g., from an ESG tool that outputs per-scenario returns).

When to use

- **ESG Integration:** Load per-scenario returns or yield curves from economic scenario generator outputs stored as separate files.
- **Stress Testing:** Combine base, stressed, and adverse scenario assumption files into a single Table for multi-scenario runs.
- **Regulatory Scenarios:** Load prescribed regulatory scenarios (e.g., IFRS17, Solvency II) from separate assumption files.

Parameters:

| Name              | Type             | Description                      | Default                                                              |
| ----------------- | ---------------- | -------------------------------- | -------------------------------------------------------------------- |
| `scenario_files`  | \`dict\[str, str | Path\]\`                         | Mapping of scenario_id -> file path                                  |
| `scenario_column` | `str`            | Name for the scenario ID column  | *required*                                                           |
| `dimensions`      | \`dict\[str, str | Dimension\]\`                    | Dimension mapping (excluding scenario, which is added automatically) |
| `value`           | `str`            | Value column name                | *required*                                                           |
| `name`            | \`str            | None\`                           | Optional table name (defaults to "from_scenarios")                   |
| `validate`        | `bool`           | Whether to validate data on load | `True`                                                               |
| `metadata`        | \`dict[str, Any] | None\`                           | Optional metadata dictionary                                         |

Returns:

| Type    | Description                                    |
| ------- | ---------------------------------------------- |
| `Table` | Table with scenario_column added to dimensions |

Examples:

Loading per-scenario rate files:

```python no_output_check
from gaspatchio.assumptions import Table

rates_table = Table.from_scenario_files(
scenario_files={
"BASE": "scenarios/BASE/rates.parquet",
"UP": "scenarios/UP/rates.parquet",
"DOWN": "scenarios/DOWN/rates.parquet",
},
scenario_column="scenario_id",
dimensions={"year": "year"},
value="forward_rate",
name="discount_rates",
)
```

### `from_scenario_template(path_template, scenario_ids, scenario_column, dimensions, value, name=None, validate=True, metadata=None)`

Create a Table from scenario files matching a path template.

Convenience method when scenario files follow a predictable naming pattern. Expands the template with each scenario ID and delegates to from_scenario_files().

When to use

- **Templated Paths:** When scenario files follow a naming convention like `scenarios/{scenario_id}/rates.parquet` or similar patterns.
- **Stochastic Scenarios:** For thousands of numbered scenarios where manually specifying each path would be impractical.
- **Convention over Configuration:** When file organization follows a predictable directory structure per scenario.

Parameters:

| Name              | Type             | Description                         | Default                                |
| ----------------- | ---------------- | ----------------------------------- | -------------------------------------- |
| `path_template`   | `str`            | Path with {scenario_id} placeholder | *required*                             |
| `scenario_ids`    | \`list[str]      | list[int]\`                         | List of scenario IDs to load           |
| `scenario_column` | `str`            | Name for the scenario ID column     | *required*                             |
| `dimensions`      | \`dict\[str, str | Dimension\]\`                       | Dimension mapping (excluding scenario) |
| `value`           | `str`            | Value column name                   | *required*                             |
| `name`            | \`str            | None\`                              | Optional table name                    |
| `validate`        | `bool`           | Whether to validate data on load    | `True`                                 |
| `metadata`        | \`dict[str, Any] | None\`                              | Optional metadata dictionary           |

Returns:

| Type    | Description                                    |
| ------- | ---------------------------------------------- |
| `Table` | Table with scenario_column added to dimensions |

Examples:

Loading files from templated paths:

```python no_output_check
from gaspatchio.assumptions import Table

#### Files: scenarios/BASE/returns.parquet, scenarios/UP/returns.parquet

returns_table = Table.from_scenario_template(
path_template="scenarios/{scenario_id}/returns.parquet",
scenario_ids=["BASE", "UP", "DOWN"],
scenario_column="scenario_id",
dimensions={"t": "t"},
value="inv_return_mth",
)
```

### `from_shocks(base_table, shocks, value_column)`

Create multiple shocked tables from a base table and shock specifications.

Takes a base assumption table and a dictionary mapping scenario IDs to lists of shocks. Returns a dictionary of Tables, one for each scenario, with the appropriate shocks applied.

When to use

- **Sensitivity Analysis:** When you need to create multiple shocked versions of an assumption table for parameter sweeps.
- **Ad-hoc Scenarios:** When scenario shocks are defined programmatically rather than loaded from files.
- **Integration with sensitivity_analysis():** The output from sensitivity_analysis() can be passed directly to this method.

Parameters:

| Name           | Type                     | Description                                       | Default    |
| -------------- | ------------------------ | ------------------------------------------------- | ---------- |
| `base_table`   | `Table`                  | The original assumption table to apply shocks to  | *required* |
| `shocks`       | `dict[str, list[Shock]]` | Mapping of scenario ID to list of shocks to apply | *required* |
| `value_column` | `str`                    | The column to apply shocks to                     | *required* |

Returns:

| Type               | Description                                                |
| ------------------ | ---------------------------------------------------------- |
| `dict[str, Table]` | Dictionary mapping scenario IDs to shocked Table instances |

Raises:

| Type         | Description                                     |
| ------------ | ----------------------------------------------- |
| `ValueError` | If value_column doesn't exist in the base table |

Examples:

Create stressed mortality tables:

````python no_output_check
from gaspatchio.assumptions import Table
from gaspatchio.scenarios.shocks import MultiplicativeShock

base_mortality = Table(...) # Load base mortality table

shocks = {
"BASE": [],
"UP": [MultiplicativeShock(factor=1.2)],
"DOWN": [MultiplicativeShock(factor=0.8)],
}

tables = Table.from_shocks(base_mortality, shocks, value_column="qx")

#### tables["BASE"], tables["UP"], tables["DOWN"] are all Table instances

```text
Integration with sensitivity_analysis():

```python no_output_check
from gaspatchio.assumptions import Table
from gaspatchio.scenarios._sensitivity import sensitivity_analysis
import polars as pl

# Create a base table
base_df = pl.DataFrame({"age": [30, 40], "rate": [0.01, 0.02]})
base_table = Table("mortality", base_df, {"age": "age"}, "rate")

shocks = sensitivity_analysis(
    table="mortality",
    shock_type="multiplicative",
    values=[0.9, 1.0, 1.1],
)

tables = Table.from_shocks(base_table, shocks, value_column="rate")
````

### `lookup(_dimensions=None, on_missing=None, **kwargs)`

Create a lookup expression using dimension names.

Generates a high-performance lookup expression that retrieves assumption values from the registered table based on provided dimension keys. The lookup is optimized for vectorized operations and integrates seamlessly with ActuarialFrame workflows for efficient model projections and calculations.

When to use

- **Model Projections:** Retrieve mortality, lapse, expense, or interest rates during actuarial model calculations and cash flow projections.
- **Dynamic Lookups:** Perform lookups where dimension values come from model point data or intermediate calculation results.
- **Multi-Dimensional Tables:** Look up values from tables with multiple dimensions like age, duration, product type, and risk class.
- **Vectorized Operations:** Execute efficient batch lookups across thousands or millions of policies in model projections.

Can be called in three ways:

1. With keyword arguments for clean dimension names: table.lookup(age=af["age"], duration=af["duration"])

1. With a dictionary for dimension names with spaces or special characters: table.lookup({"policy duration": af["policy_duration_as_int"]})

1. Or both combined: table.lookup({"policy duration": af["duration"]}, age=af["age"])

The returned expression cooperates with `af` columns in either operand order: `table.lookup(...) * af.pols` and `af.pols * table.lookup(...)` agree, including the list-column operator shims (`**` on per-period columns). That guarantee covers *operators* only — an Expr method call (`.clip()`, `.fill_null()`, …) returns a plain polars expression, after which a proxy operand raises again. For method chains, assign the lookup to a column first (`af.mort_rate = table.lookup(...)`) and continue from the column.

Parameters:

| Name          | Type             | Description | Default       |
| ------------- | ---------------- | ----------- | ------------- |
| `_dimensions` | \`dict\[str, str | Expr        | ColumnProxy\] |
| `on_missing`  | \`str            | float       | None\`        |
| `**kwargs`    | \`str            | Expr        | ColumnProxy\` |

Returns:

| Type   | Description                      |
| ------ | -------------------------------- |
| `Expr` | Polars expression for the lookup |

##### Examples:

**Scalar Example: Simple Mortality Lookup**

```python
from gaspatchio.assumptions import Table
from gaspatchio import ActuarialFrame
import polars as pl

# Create mortality table
mortality_data = pl.DataFrame(
    {
        "age": [30, 35, 40, 45, 50],
        "mortality_rate": [0.001, 0.002, 0.004, 0.008, 0.015],
    }
)

mortality_table = Table(
    name="mortality_std",
    source=mortality_data,
    dimensions={"age": "age"},
    value="mortality_rate",
)

# Create model data and perform lookup
model_data = {
    "policy_id": ["P001", "P002", "P003"],
    "current_age": [35, 40, 50],
}
af = ActuarialFrame(model_data)

# Lookup mortality rates
af.mortality_rate = mortality_table.lookup(age=af.current_age)

print(af.collect())
```

```text
shape: (3, 3)
┌───────────┬─────────────┬────────────────┐
│ policy_id ┆ current_age ┆ mortality_rate │
│ ---       ┆ ---         ┆ ---            │
│ str       ┆ i64         ┆ f64            │
╞═══════════╪═════════════╪════════════════╡
│ P001      ┆ 35          ┆ 0.002          │
│ P002      ┆ 40          ┆ 0.004          │
│ P003      ┆ 50          ┆ 0.015          │
└───────────┴─────────────┴────────────────┘
```

**Vector Example: Multi-Dimensional Lapse Lookup**

```python
from gaspatchio.assumptions import Table
from gaspatchio import ActuarialFrame
import polars as pl

# Create multi-dimensional lapse table
lapse_data = pl.DataFrame({
    "duration": [1, 1, 2, 2, 3, 3],
    "product_type": ["TERM", "WL", "TERM", "WL", "TERM", "WL"],
    "lapse_rate": [0.05, 0.03, 0.08, 0.05, 0.12, 0.07]
})

lapse_table = Table(
    name="lapse_rates",
    source=lapse_data,
    dimensions={"duration": "duration", "product_type": "product_type"},
    value="lapse_rate"
)

# Create model points with policy data
model_points = {
    "policy_id": ["P001", "P002", "P003", "P004"],
    "product_code": ["TERM", "WL", "TERM", "WL"],
    "policy_year": [1, 2, 3, 1]
}
af = ActuarialFrame(model_points)

# Lookup lapse rates using multiple dimensions
af.lapse_rate = lapse_table.lookup(
    duration=af.policy_year,
    product_type=af.product_code
)

print(af.collect())
```

```text
shape: (4, 4)
┌───────────┬──────────────┬─────────────┬────────────┐
│ policy_id ┆ product_code ┆ policy_year ┆ lapse_rate │
│ ---       ┆ ---          ┆ ---         ┆ ---        │
│ str       ┆ str          ┆ i64         ┆ f64        │
╞═══════════╪══════════════╪═════════════╪════════════╡
│ P001      ┆ TERM         ┆ 1           ┆ 0.05       │
│ P002      ┆ WL           ┆ 2           ┆ 0.05       │
│ P003      ┆ TERM         ┆ 3           ┆ 0.12       │
│ P004      ┆ WL           ┆ 1           ┆ 0.03       │
└───────────┴──────────────┴─────────────┴────────────┘
```

### `source_sha()`

SHA-256 over `canonical_form` bytes; stable for the same content + name.

The single content hash an auditor can reproduce from the same input data. Identical Tables produce identical SHAs; any change to name, dimensions, value column, or row content changes the SHA. Combine with `ScenarioRun.source_sha()` to attest that an SCR run used a specific input table.

When to use

- **Audit sidecar:** Embedded under the plan's `canonical_form`/`source_sha` chain so the input data is identifiable from the run record alone.
- **Pre-run validation:** Compare against a known-good SHA before running production batches to catch silent drift in assumption files.
- **Cross-team reproducibility:** Two analysts loading the same parquet file produce the same SHA regardless of their load order.

Returns:

| Type  | Description                                                |
| ----- | ---------------------------------------------------------- |
| `str` | 64-character lowercase hexadecimal SHA-256 digest prefixed |
| `str` | with sha256:.                                              |

##### Examples:

```python
import polars as pl
from gaspatchio.assumptions import Table

mortality = Table(
    name="mortality",
    source=pl.DataFrame({"age": [30, 31], "rate": [0.001, 0.0012]}),
    dimensions={"age": "age"},
    value="rate",
)

sha = mortality.source_sha()
print(sha.startswith("sha256:"))
```

```text
True
```

### `to_dataframe()`

Export the complete table as a DataFrame.

Returns the complete processed assumption table as a Polars DataFrame, including all key columns and the value column after dimension processing. Useful for data inspection, validation, and integration with external systems.

When to use

- **Data Inspection:** Export table data for validation, quality checks, and manual review of assumption values.
- **Integration:** Export assumption data for use in external systems, reporting tools, or alternative calculation engines.
- **Debugging:** Examine processed table structure and data after dimension transformations and validation.

Returns:

| Type        | Description                                          |
| ----------- | ---------------------------------------------------- |
| `DataFrame` | pl.DataFrame: Complete table with all processed data |

##### Examples:

```python
from gaspatchio.assumptions import Table
import polars as pl

data = pl.DataFrame({"age": [30, 35], "rate": [0.001, 0.002]})
table = Table("test", data, {"age": "age"}, "rate")

df = table.to_dataframe()
print(f"Exported {len(df)} rows")
```

### `validate_lookup(_dimensions=None, **kwargs)`

Validate a lookup configuration without executing.

Checks that a lookup configuration provides all required dimensions and that dimension names match the table's configuration. Useful for validating lookup calls before execution and catching errors early.

When to use

- **Error Prevention:** Validate lookup configurations before execution to catch missing or invalid dimensions early.
- **Dynamic Validation:** Check programmatically generated lookup calls for correctness in complex model workflows.
- **Testing:** Validate lookup configurations in unit tests without executing expensive lookup operations.

Parameters:

| Name          | Type             | Description                                 | Default       |
| ------------- | ---------------- | ------------------------------------------- | ------------- |
| `_dimensions` | \`dict\[str, str | Expr                                        | ColumnProxy\] |
| `**kwargs`    |                  | Dimension name to column/expression mapping | `{}`          |

Raises:

| Type         | Description                           |
| ------------ | ------------------------------------- |
| `ValueError` | If dimension configuration is invalid |

##### Examples:

```python
from gaspatchio.assumptions import Table
import polars as pl

data = pl.DataFrame({"age": [30, 35], "rate": [0.001, 0.002]})

table = Table("test", data, {"age": "age"}, "rate")

# Valid lookup - no error
table.validate_lookup(age="current_age")

# Invalid lookup - raises ValueError
try:
    table.validate_lookup(invalid_dim="some_col")
except ValueError as e:
    print(f"Validation error: {e}")
```

### `with_shock(shock, name=None)`

Apply a shock to create a modified copy of this table.

Creates a new Table with the shock applied to the value column. The original table is unchanged. This enables scenario analysis by creating stressed versions of assumption tables.

When to use

- **Stress Testing:** Create stressed assumption tables for regulatory capital calculations and risk analysis.
- **Sensitivity Analysis:** Generate tables with parameter variations to understand model sensitivity to assumptions.
- **Ad-hoc Scenarios:** Create one-off shocked tables without needing to load separate scenario files.

Parameters:

| Name    | Type    | Description                                                          | Default                                                            |
| ------- | ------- | -------------------------------------------------------------------- | ------------------------------------------------------------------ |
| `shock` | `Shock` | Shock specification to apply (Multiplicative, Additive, or Override) | *required*                                                         |
| `name`  | \`str   | None\`                                                               | Optional name for the shocked table (defaults to original_shocked) |

Returns:

| Type    | Description                   |
| ------- | ----------------------------- |
| `Table` | New Table with shocked values |

##### Examples:

**Stress testing mortality:**

````python no_output_check
from gaspatchio.assumptions import Table
from gaspatchio.scenarios.shocks import MultiplicativeShock
import polars as pl

mortality_data = pl.DataFrame({"age": [30, 40], "qx": [0.001, 0.002]})
mortality = Table("mortality", mortality_data, {"age": "age"}, "qx")

#### Create 20% stressed version

shocked = mortality.with_shock(MultiplicativeShock(factor=1.2))

```text
**Adding basis points to rates:**

```python no_output_check
from gaspatchio.assumptions import Table
from gaspatchio.scenarios.shocks import AdditiveShock
import polars as pl

rates_data = pl.DataFrame({"term": [1, 2], "rate": [0.05, 0.06]})
rates_table = Table("rates", rates_data, {"term": "term"}, "rate")

# Add 50bps to discount rates
stressed_rates = rates_table.with_shock(AdditiveShock(delta=0.005))
````

## TableBuilder

## `gaspatchio.assumptions._builder.TableBuilder`

Fluent builder for complex table configurations.

### `build()`

Build the Table object from the configured builder.

Returns:

| Type    | Description               |
| ------- | ------------------------- |
| `Table` | Configured Table instance |

Raises:

| Type         | Description                                          |
| ------------ | ---------------------------------------------------- |
| `ValueError` | If source is not set or no dimensions are configured |

### `copy()`

Create a copy of this builder.

Returns:

| Type           | Description                              |
| -------------- | ---------------------------------------- |
| `TableBuilder` | New TableBuilder with same configuration |

### `from_source(source)`

Set the data source for the table.

Parameters:

| Name     | Type  | Description | Default     |
| -------- | ----- | ----------- | ----------- |
| `source` | \`str | Path        | DataFrame\` |

Returns:

| Type           | Description       |
| -------------- | ----------------- |
| `TableBuilder` | Self for chaining |

### `reset()`

Reset the builder to initial state (keeping only the name).

Returns:

| Type           | Description       |
| -------------- | ----------------- |
| `TableBuilder` | Self for chaining |

### `with_categorical_dimension(name, value, dimension_name=None)`

Add a categorical dimension with a constant value.

Parameters:

| Name             | Type  | Description                       | Default                                       |
| ---------------- | ----- | --------------------------------- | --------------------------------------------- |
| `name`           | `str` | Dimension name                    | *required*                                    |
| `value`          | `Any` | Constant value for this dimension | *required*                                    |
| `dimension_name` | \`str | None\`                            | Optional custom name for the dimension column |

Returns:

| Type           | Description       |
| -------------- | ----------------- |
| `TableBuilder` | Self for chaining |

### `with_computed_dimension(name, expression, alias=None)`

Add a computed dimension from an expression.

Parameters:

| Name         | Type   | Description                                | Default                                |
| ------------ | ------ | ------------------------------------------ | -------------------------------------- |
| `name`       | `str`  | Dimension name                             | *required*                             |
| `expression` | `Expr` | Polars expression to compute the dimension | *required*                             |
| `alias`      | \`str  | None\`                                     | Optional alias for the computed column |

Returns:

| Type           | Description       |
| -------------- | ----------------- |
| `TableBuilder` | Self for chaining |

### `with_data_dimension(name, column, rename_to=None, dtype=None)`

Add a data dimension that maps directly from a column.

Parameters:

| Name        | Type       | Description        | Default                           |
| ----------- | ---------- | ------------------ | --------------------------------- |
| `name`      | `str`      | Dimension name     | *required*                        |
| `column`    | `str`      | Source column name | *required*                        |
| `rename_to` | \`str      | None\`             | Optional rename for the dimension |
| `dtype`     | \`DataType | None\`             | Optional data type conversion     |

Returns:

| Type           | Description       |
| -------------- | ----------------- |
| `TableBuilder` | Self for chaining |

### `with_dimension(name, dimension)`

Add a pre-configured dimension object.

Parameters:

| Name        | Type        | Description      | Default    |
| ----------- | ----------- | ---------------- | ---------- |
| `name`      | `str`       | Dimension name   | *required* |
| `dimension` | `Dimension` | Dimension object | *required* |

Returns:

| Type           | Description       |
| -------------- | ----------------- |
| `TableBuilder` | Self for chaining |

### `with_melt_dimension(name, columns, overflow=None, fill=None)`

Add a melt dimension that transforms wide columns to long format.

Parameters:

| Name       | Type        | Description             | Default                    |
| ---------- | ----------- | ----------------------- | -------------------------- |
| `name`     | `str`       | Dimension name          | *required*                 |
| `columns`  | `list[str]` | List of columns to melt | *required*                 |
| `overflow` | \`Any       | None\`                  | Optional overflow strategy |
| `fill`     | \`Any       | None\`                  | Optional fill strategy     |

Returns:

| Type           | Description       |
| -------------- | ----------------- |
| `TableBuilder` | Self for chaining |

### `with_value_column(name)`

Set the name of the value column.

Parameters:

| Name   | Type  | Description       | Default    |
| ------ | ----- | ----------------- | ---------- |
| `name` | `str` | Value column name | *required* |

Returns:

| Type           | Description       |
| -------------- | ----------------- |
| `TableBuilder` | Self for chaining |

## Registry Functions

### list_tables

List all registered assumption tables.

Retrieves the names of all assumption tables that have been registered with the framework. Essential for model inventory management, debugging lookup failures, and ensuring all required tables are available before running actuarial projections or model validations.

When to use

- **Model Validation:** Check that all required assumption tables are loaded before starting model calculations or projections.
- **Debugging:** Troubleshoot lookup failures by verifying table registration status and identifying missing tables.
- **Model Inventory:** Generate reports of available assumption tables for model documentation and governance processes.
- **Dynamic Configuration:** Build dynamic model configurations that adapt based on available assumption tables.

Returns:

| Type        | Description                                                |
| ----------- | ---------------------------------------------------------- |
| `list[str]` | list\[str\]: List of table names that have been registered |

### Examples:

**Scalar Example: Basic Table Listing**

```python
from gaspatchio.assumptions import Table, list_tables
import polars as pl

# Register some assumption tables
mortality_data = pl.DataFrame(
    {"age": [30, 40, 50], "mortality_rate": [0.001, 0.004, 0.015]}
)

lapse_data = pl.DataFrame({"duration": [1, 2, 3], "lapse_rate": [0.05, 0.08, 0.12]})

Table(
    name="mortality_list_ex",
    source=mortality_data,
    dimensions={"age": "age"},
    value="mortality_rate",
)
Table(
    name="lapse_list_ex",
    source=lapse_data,
    dimensions={"duration": "duration"},
    value="lapse_rate",
)

# Check that tables were registered
tables = list_tables()
print("mortality_list_ex registered:", "mortality_list_ex" in tables)
print("lapse_list_ex registered:", "lapse_list_ex" in tables)
```

```text
mortality_list_ex registered: True
lapse_list_ex registered: True
```

**Vector Example: Model Validation Workflow**

```python
from gaspatchio.assumptions import Table, list_tables
import polars as pl

# Define required tables for a term life model
required_tables = [
    "mortality_validation_ex",
    "lapse_validation_ex",
    "expense_validation_ex",
    "interest_validation_ex"
]

# Register some tables (simulating partial loading)
mortality_data = pl.DataFrame({
    "age": [25, 30, 35, 40],
    "rate": [0.0008, 0.001, 0.0015, 0.0025]
})

lapse_data = pl.DataFrame({
    "duration": [1, 2, 3, 4],
    "rate": [0.05, 0.08, 0.10, 0.12]
})

Table(name="mortality_validation_ex", source=mortality_data,
      dimensions={"age": "age"}, value="rate")
Table(name="lapse_validation_ex", source=lapse_data,
      dimensions={"duration": "duration"}, value="rate")

# Validate model readiness
available_tables = list_tables()
missing_tables = [table for table in required_tables
                 if table not in available_tables]

print("Loaded tables:", ["mortality_validation_ex", "lapse_validation_ex"])
print("Missing tables:", missing_tables)
print(f"⚠️  Model not ready - missing {len(missing_tables)} tables")
```

```text
Loaded tables: ['mortality_validation_ex', 'lapse_validation_ex']
Missing tables: ['expense_validation_ex', 'interest_validation_ex']
⚠️  Model not ready - missing 2 tables
```

### list_tables_with_metadata

List all assumption tables that have metadata stored.

Returns a dictionary mapping table names to their stored metadata for all tables that were registered with metadata. Useful for generating comprehensive model documentation, conducting data lineage analysis, and ensuring proper governance over assumption tables used in actuarial models.

When to use

- **Documentation Generation:** Create comprehensive model documentation showing all assumption tables with their descriptions and sources.
- **Governance Reporting:** Generate reports for regulatory compliance showing data lineage, validation status, and review dates.
- **Quality Assurance:** Identify tables missing critical metadata like effective dates, validation status, or business descriptions.
- **Model Inventory:** Maintain centralized inventory of all assumption tables with their business context and technical specifications.

Returns:

| Name   | Type                        | Description                                      |
| ------ | --------------------------- | ------------------------------------------------ |
| `dict` | `dict[str, dict[str, Any]]` | Dictionary mapping table names to their metadata |

### Examples:

**Scalar Example: Basic Metadata Listing**

```python
from gaspatchio.assumptions import Table, list_tables_with_metadata
import polars as pl

# Register tables with rich metadata
mortality_data = pl.DataFrame({"age": [30, 40, 50], "rate": [0.001, 0.004, 0.015]})

Table(
    name="mortality_meta_ex1",
    source=mortality_data,
    dimensions={"age": "age"},
    value="rate",
    metadata={
        "description": "Base mortality rates for healthy lives",
        "source": "Company Experience Study 2023",
        "effective_date": "2024-01-01",
    },
)

# Check table has metadata
tables_metadata = list_tables_with_metadata()
print(f"mortality_meta_ex1 has metadata: {'mortality_meta_ex1' in tables_metadata}")
```

```text
mortality_meta_ex1 has metadata: True
```

**Vector Example: Model Documentation Report**

```python
from gaspatchio.assumptions import Table, list_tables_with_metadata
import polars as pl

# Register multiple tables with metadata
mortality_df = pl.DataFrame({
    "age": [30, 35, 40],
    "rate": [0.001, 0.002, 0.004]
})

lapse_df = pl.DataFrame({
    "duration": [1, 2, 3],
    "rate": [0.05, 0.08, 0.12]
})

# Create tables with metadata
Table(
    name="mortality_meta_ex2",
    source=mortality_df,
    dimensions={"age": "age"},
    value="rate",
    metadata={"source": "2017 CSO", "version": "v2.1"}
)

Table(
    name="lapse_meta_ex2",
    source=lapse_df,
    dimensions={"duration": "duration"},
    value="rate",
    metadata={"source": "Company Study", "quality": "High"}
)

# Check tables with metadata
tables_meta = list_tables_with_metadata()
has_mortality = "mortality_meta_ex2" in tables_meta
has_lapse = "lapse_meta_ex2" in tables_meta
print("Found 2 tables with metadata")
print(f"mortality_meta_ex2 registered: {has_mortality}")
print(f"lapse_meta_ex2 registered: {has_lapse}")
```

```text
Found 2 tables with metadata
mortality_meta_ex2 registered: True
lapse_meta_ex2 registered: True
```

### get_table_metadata

Retrieve metadata for a registered assumption table.

Fetches stored metadata for an assumption table that was registered with the framework. Metadata includes information like table descriptions, data sources, validation rules, effective dates, and business context that actuaries need for model documentation and compliance reporting.

When to use

- **Model Documentation:** Retrieve table descriptions, sources, and business context for automated model documentation generation.
- **Audit Trails:** Access metadata for regulatory compliance and audit trails showing table lineage and validation status.
- **Data Validation:** Check table metadata before performing lookups to ensure data quality and appropriateness for calculations.
- **Model Versioning:** Track assumption table versions and effective dates for model change management and rollback procedures.

Parameters:

| Name         | Type  | Description                           | Default    |
| ------------ | ----- | ------------------------------------- | ---------- |
| `table_name` | `str` | Name of the table to get metadata for | *required* |

Returns:

| Type             | Description |
| ---------------- | ----------- |
| \`dict[str, Any] | None\`      |

### Examples:

**Scalar Example: Basic Metadata Retrieval**

```python
from gaspatchio.assumptions import Table, get_table_metadata
import polars as pl

# Create and register a mortality table with metadata
mortality_data = pl.DataFrame(
    {
        "age": [30, 35, 40, 45, 50],
        "mortality_rate": [0.001, 0.002, 0.004, 0.008, 0.015],
    }
)

mortality_table = Table(
    name="mortality_2023",
    source=mortality_data,
    dimensions={"age": "age"},
    value="mortality_rate",
    metadata={
        "description": "Standard mortality rates for term life insurance",
        "source": "Industry Standard Tables 2023",
        "effective_date": "2023-01-01",
        "validation_status": "approved",
    },
)

# Retrieve metadata
metadata = get_table_metadata("mortality_2023")
print(metadata)
```

```text
{'description': 'Standard mortality rates for term life insurance', 'source': 'Industry Standard Tables 2023', 'effective_date': '2023-01-01', 'validation_status': 'approved'}
```

**Vector Example: Metadata for Model Documentation**

```python
from gaspatchio.assumptions import Table, get_table_metadata
import polars as pl

# Create multiple assumption tables with rich metadata
tables_config = [
    {
        "name": "lapse_rates_term",
        "data": pl.DataFrame({
            "duration": [1, 2, 3, 4, 5],
            "lapse_rate": [0.05, 0.08, 0.12, 0.15, 0.18]
        }),
        "metadata": {
            "description": "Lapse rates for term life products",
            "business_unit": "Individual Life",
            "last_updated": "2023-12-01",
            "data_quality": "high"
        }
    },
    {
        "name": "expense_rates",
        "data": pl.DataFrame({
            "year": [1, 2, 3],
            "expense_rate": [150.0, 25.0, 15.0]
        }),
        "metadata": {
            "description": "Annual expense rates per policy",
            "currency": "USD",
            "inflation_adjusted": True,
            "review_frequency": "quarterly"
        }
    }
]

# Register lapse rates table
Table(
    name="lapse_rates_term",
    source=tables_config[0]["data"],
    dimensions={"duration": "duration"},
    value="lapse_rate",
    metadata=tables_config[0]["metadata"]
)

# Register expense rates table
Table(
    name="expense_rates",
    source=tables_config[1]["data"],
    dimensions={"year": "year"},
    value="expense_rate",
    metadata=tables_config[1]["metadata"]
)

# Check metadata count
print(f"Registered {len([get_table_metadata('lapse_rates_term'), get_table_metadata('expense_rates')])} tables with metadata")
```

```text
Registered 2 tables with metadata
```

# Conditionals API

## when

Start a conditional expression chain.

Excel-style IF() function with method chaining for multiple conditions. Provides intuitive if/elif/else logic for actuarial calculations. Automatically handles both scalar columns and list columns (projections) with proper broadcasting.

**Supported in both debug and optimize modes** - conditionals with list columns work seamlessly in either execution mode.

When to use

- **Age-Based Pricing:** Apply different premium rates, mortality factors, or underwriting classes based on policyholder age brackets.
- **Maturity Events:** Identify when policies mature by comparing projection month against policy term, zeroing cash flows after maturity.
- **Premium Holidays:** Suspend premium collection for specific months or conditions, such as grace periods or payment holidays.
- **Commission Schedules:** Calculate tiered commission rates based on policy value, product type, or sales channel.
- **Benefit Triggers:** Activate guaranteed minimum benefits, death benefits, or surrender values when specific conditions are met.
- **Underwriting Rules:** Implement automated underwriting decisions based on sum assured, age, and other risk factors.

Parameters:

| Name        | Type  | Description                            | Default    |
| ----------- | ----- | -------------------------------------- | ---------- |
| `condition` | `Any` | Boolean expression (e.g., af.age > 65) | *required* |

Returns:

| Type               | Description                                            |
| ------------------ | ------------------------------------------------------ |
| `ConditionalProxy` | ConditionalProxy for chaining .then() and .otherwise() |

### Examples:

**Scalar Example: Age-Based Rate Classification**

```python
from gaspatchio import ActuarialFrame, when

data = {
    "policy_id": ["P001", "P002", "P003", "P004"],
    "age": [35, 55, 68, 72],
}
af = ActuarialFrame(data)

af.rate_class = when(af.age > 65).then("senior").otherwise("standard")

print(af.collect())
```

```text
shape: (4, 3)
┌───────────┬─────┬────────────┐
│ policy_id ┆ age ┆ rate_class │
│ ---       ┆ --- ┆ ---        │
│ str       ┆ i64 ┆ str        │
╞═══════════╪═════╪════════════╡
│ P001      ┆ 35  ┆ standard   │
│ P002      ┆ 55  ┆ standard   │
│ P003      ┆ 68  ┆ senior     │
│ P004      ┆ 72  ┆ senior     │
└───────────┴─────┴────────────┘
```

**Vector Example: Maturity Detection with List Broadcasting**

```python
from gaspatchio import ActuarialFrame, when

data = {
    "policy_id": ["P001", "P002"],
    "month": [
        [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12],
        [
            0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12,
            13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24
        ]
    ],
    "policy_term_years": [1, 2],
    "pols_if": [
        [1000, 998, 996, 994, 992, 990, 988, 986, 984, 982, 980, 978, 976],
        [
            1000, 998, 996, 994, 992, 990, 988, 986, 984, 982, 980, 978,
            976, 974, 972, 970, 968, 966, 964, 962, 960, 958, 956, 954, 952
        ]
    ],
}
af = ActuarialFrame(data)

af.pols_maturity = (
    when(af.month == af.policy_term_years * 12)
    .then(af.pols_if)
    .otherwise(0)
)

print(af.collect())
```

```text
shape: (2, 5)
┌───────────┬──────────────┬──────────┬───────────────┐
│ policy_id ┆ month        ┆ pols_if  ┆ pols_maturity │
│ ---       ┆ ---          ┆ ---      ┆ ---           │
│ str       ┆ list[i64]    ┆ list[... ┆ list[i64]     │
╞═══════════╪══════════════╪══════════╪═══════════════╡
│ P001      ┆ [0, 1, … 12] ┆ [1000... ┆ [0, 0, … 976] │
│ P002      ┆ [0, 1, … 24] ┆ [1000... ┆ [0, 0, … 952] │
└───────────┴──────────────┴──────────┴───────────────┘
```

**List Broadcasting Behavior**

When list columns are involved, the framework automatically broadcasts scalar values across all elements in the list. In the maturity example above:

- `af.month` is a list column (projection months 0-12 and 0-24)
- `af.policy_term_years * 12` broadcasts the scalar calculation to each month
- The condition is evaluated element-wise within each list
- `af.pols_if` (then value) and `0` (otherwise value) are applied element-wise
- Result: maturity value appears only at the matching month, zeros elsewhere

## ConditionalProxy

Represents an in-progress conditional expression chain.

This class builds up when/then chains and completes them with otherwise(). It automatically routes to the list_conditional Rust plugin for list columns or standard Polars when/then/otherwise for scalar columns.

## `needs_list_broadcasting()`

Check if this conditional requires list broadcasting.

Returns:

| Type   | Description                                                    |
| ------ | -------------------------------------------------------------- |
| `bool` | True if any columns involved are list columns, False otherwise |

## `otherwise(value)`

Complete conditional chain with default value.

Finalizes the conditional expression by providing the value to use when none of the preceding conditions evaluate to true. This method is required - a conditional expression cannot be used without calling `.otherwise()`. Automatically detects and handles list broadcasting for projection calculations.

When to use

- **Default Rate:** Provide standard rate when age doesn't match any premium tiers or risk categories.
- **Zero After Event:** Set cash flows to zero for all months after maturity, surrender, or death events occur.
- **Fallback Values:** Apply baseline commission rates, default mortality assumptions, or standard policy terms when special conditions aren't met.
- **Maintain Status Quo:** Keep existing premium, benefit, or reserve values unchanged when update conditions don't apply.

Parameters:

| Name    | Type  | Description                                                                                                                                                                                | Default    |
| ------- | ----- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------- |
| `value` | `Any` | Default value when no conditions match. Can be a literal value, column reference, or computed expression. For list columns, this value is broadcast element-wise across all list elements. | *required* |

Returns:

| Type              | Description                                                         |
| ----------------- | ------------------------------------------------------------------- |
| `ExpressionProxy` | ExpressionProxy wrapping the complete conditional expression, ready |
| `ExpressionProxy` | for assignment to a column.                                         |

#### Examples:

**Scalar Example: Underwriting Classification**

```python
from gaspatchio import ActuarialFrame, when

data = {
    "policy_id": ["P001", "P002", "P003", "P004", "P005", "P006"],
    "age": [25, 42, 55, 68, 73, 45],
    "sum_assured": [100000, 250000, 500000, 150000, 300000, 600000],
}
af = ActuarialFrame(data)

af.underwriting_class = (
    when(af.sum_assured > 500000)
    .then("refer_underwriting")
    .when(af.age > 65)
    .then("senior_standard")
    .when(af.age < 35)
    .then("young_preferred")
    .otherwise("standard")
)

print(af.collect())
```

```text
shape: (6, 4)
┌───────────┬─────┬─────────────┬────────────────────┐
│ policy_id ┆ age ┆ sum_assured ┆ underwriting_class │
│ ---       ┆ --- ┆ ---         ┆ ---                │
│ str       ┆ i64 ┆ i64         ┆ str                │
╞═══════════╪═════╪═════════════╪════════════════════╡
│ P001      ┆ 25  ┆ 100000      ┆ young_preferred    │
│ P002      ┆ 42  ┆ 250000      ┆ standard           │
│ P003      ┆ 55  ┆ 500000      ┆ standard           │
│ P004      ┆ 68  ┆ 150000      ┆ senior_standard    │
│ P005      ┆ 73  ┆ 300000      ┆ senior_standard    │
│ P006      ┆ 45  ┆ 600000      ┆ refer_underwriting │
└───────────┴─────┴─────────────┴────────────────────┘
```

**List Broadcasting Behavior**

The `.otherwise()` method automatically detects when list columns are involved and applies the default value element-wise. If the otherwise value is a scalar (like `0` or `100.0`), it's broadcast to match the length of each list. If the otherwise value is itself a list column, elements are matched one-to-one.

This enables patterns like:

- Zeroing cash flows after maturity: `.otherwise(0)`
- Maintaining baseline premiums: `.otherwise(af.base_premium)`
- Default growth rates: `.otherwise(0.03)` broadcasts to all months

## `then(value)`

Specify value when condition is true.

Defines the result value for when the preceding condition evaluates to true. Must be followed by either another `.when()` for chained conditions or `.otherwise()` to complete the expression. Works with scalar values, column references, or computed expressions.

Parameters:

| Name    | Type  | Description                                                                                                                                                                                                                                             | Default    |
| ------- | ----- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- |
| `value` | `Any` | Value to return when condition matches. Can be a literal value (number, string, etc.), a column reference (af.column_name), or a computed expression (af.premium * 1.1). For list columns, values are applied element-wise with automatic broadcasting. | *required* |

Returns:

| Type               | Description                                          |
| ------------------ | ---------------------------------------------------- |
| `ConditionalProxy` | Self for chaining more .when() or final .otherwise() |

#### Examples:

**Scalar Example: Multi-Tier Premium Rates**

```python
from gaspatchio import ActuarialFrame, when

data = {
    "policy_id": ["P001", "P002", "P003", "P004", "P005"],
    "age": [25, 42, 55, 68, 73],
}
af = ActuarialFrame(data)

af.premium_rate = (
    when(af.age < 35)
    .then(0.0015)
    .when(af.age < 50)
    .then(0.0025)
    .when(af.age < 65)
    .then(0.0040)
    .otherwise(0.0065)
)

print(af.collect())
```

```text
shape: (5, 3)
┌───────────┬─────┬──────────────┐
│ policy_id ┆ age ┆ premium_rate │
│ ---       ┆ --- ┆ ---          │
│ str       ┆ i64 ┆ f64          │
╞═══════════╪═════╪══════════════╡
│ P001      ┆ 25  ┆ 0.0015       │
│ P002      ┆ 42  ┆ 0.0025       │
│ P003      ┆ 55  ┆ 0.004        │
│ P004      ┆ 68  ┆ 0.0065       │
│ P005      ┆ 73  ┆ 0.0065       │
└───────────┴─────┴──────────────┘
```

**Vector Example: Premium Holiday**

```python
from gaspatchio import ActuarialFrame, when

data = {
    "policy_id": ["P001"],
    "month": [[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]],
    "premium_holiday_month": [6],
    "base_premium": [100.0],
}
af = ActuarialFrame(data)

af.premium_due = (
    when(af.month == af.premium_holiday_month)
    .then(0.0)
    .otherwise(af.base_premium)
)

print(af.collect())
```

```text
shape: (1, 5)
┌───────────┬──────────────┬──────────┬─────────────────────────┐
│ policy_id ┆ month        ┆ base...  ┆ premium_due             │
│ ---       ┆ ---          ┆ ---      ┆ ---                     │
│ str       ┆ list[i64]    ┆ f64      ┆ list[f64]               │
╞═══════════╪══════════════╪══════════╪═════════════════════════╡
│ P001      ┆ [0, 1, … 12] ┆ 100.0    ┆ [100.0, 100.0, … 100.0] │
└───────────┴──────────────┴──────────┴─────────────────────────┘
```

## `when(condition)`

Add another condition (elif behavior).

Parameters:

| Name        | Type  | Description                     | Default    |
| ----------- | ----- | ------------------------------- | ---------- |
| `condition` | `Any` | Additional condition expression | *required* |

Returns:

| Type               | Description               |
| ------------------ | ------------------------- |
| `ConditionalProxy` | Self for chaining .then() |

## ConditionExpression

## `gaspatchio.column.condition_expression.ConditionExpression`

Wraps a comparison expression with metadata for list_conditional plugin.

Created by ColumnProxy/ExpressionProxy comparison operators (**eq**, **lt**, etc.). Stores the operator type and operands needed to call list_conditional Rust plugin.

This class enables the elimination of EXPLODE/GROUP_BY patterns by tracking comparison metadata at creation time, allowing ConditionalProxy to call the plugin directly instead of using the expensive EXPLODE pattern.

Attributes:

| Name       | Type | Description                                                |
| ---------- | ---- | ---------------------------------------------------------- |
| `_expr`    |      | The Polars comparison expression (lazy, for compatibility) |
| `_parent`  |      | Parent ActuarialFrame for context                          |
| `operator` |      | Comparison operator ("eq", "ne", "lt", "lte", "gt", "gte") |
| `left`     |      | Left operand expression                                    |
| `right`    |      | Right operand expression                                   |

### `left_shape`

Resolved shape of the left operand.

### `right_shape`

Resolved shape of the right operand.

### `shape`

Resolved shape of this comparison — the max of operand shapes.

### `normalize_for_list_path()`

Return `(left, right, operator)` ordered for `list_conditional`.

The `list_conditional` plugin requires the list-typed operand on `left` ("left must be List dtype"). User code can produce commuted predicates like `(scalar) == af.list_col` where the list lands on the right; this swaps operands and inverts the operator so the plugin contract holds. Predicates already in canonical form pass through unchanged.

List column limitation

Multiple chained `.when()` calls (if/elif/else pattern) are not supported when working with list (vector) columns. Use separate conditionals or combine conditions with the `&` operator instead.

# Curves API

For the actuarial framing — when to build with `from_zero_rates` vs `from_par_rates`, when to use parallel vs key-rate shifts, how Curves compose with Schedules — see [Curves](https://gaspatchio.dev/0.9.0/concepts/curves/index.md).

## Curve

## `gaspatchio.curves._curve.Curve`

Typed term-structure curve.

Construct via :meth:`from_zero_rates` or :meth:`from_par_rates`. Direct construction is intentionally awkward — use the classmethods.

### `canonical_form()`

Return the JSON-encodable canonical form of this Curve.

For knot-based curves the form is identical to previous versions (keys: `kind`, `tenors`, `rates`, `day_count`, `interpolation`). For parametric curves an additional `parametric` sub-dict is included with `kind` and all parameters, while `tenors` and `rates` are empty lists (backward-compatible: knot curves with no `parametric` field produce the exact same bytes as before this change).

Returns:

| Type                | Description                                               |
| ------------------- | --------------------------------------------------------- |
| `dict[str, object]` | A JSON-serialisable dict uniquely identifying this curve. |

Examples:

```pycon
>>> c = Curve.from_zero_rates(tenors=[1.0, 5.0], rates=[0.03, 0.04])
>>> c.canonical_form()["kind"]
'Curve'
>>> isinstance(c.canonical_form()["tenors"], list)
True
>>> c2 = Curve.from_svensson(
...     b0=0.04, b1=-0.01, b2=0.005, b3=0.002, tau1=1.5, tau2=10.0
... )
>>> c2.canonical_form()["parametric"]["kind"]
'svensson'
```

### `discount_factor(t)`

Annually compounded discount factor: `DF(t) = (1 + r(t))^(-t)`.

Discounting is annually compounded; continuously compounded (`exp(-r*t)`) is not yet supported. Two curves with identical rate grids but different compounding frequencies would produce meaningfully different DFs — the choice is canonical and not user-configurable.

Supported domain is `t > 0` for `log_linear` and `smith_wilson`; for those methods an out-of-domain `t <= 0` yields `NaN`. For any method, a non-finite `t` (NaN or ±inf) yields `NaN` — the out-of-domain rate propagates through `(1 + NaN)^(-t) = NaN`.

Parameters:

| Name | Type        | Description                                                                                                                    | Default    |
| ---- | ----------- | ------------------------------------------------------------------------------------------------------------------------------ | ---------- |
| `t`  | `TimeInput` | Year fraction(s) at which to evaluate the discount factor. Accepts float, int, list[float], np.ndarray, pl.Series, or pl.Expr. | *required* |

Returns:

| Type    | Description |
| ------- | ----------- |
| \`float | list[float] |

Raises:

| Type        | Description                             |
| ----------- | --------------------------------------- |
| `TypeError` | If t is not one of the supported types. |

Examples:

```pycon
>>> from gaspatchio.curves import Curve
>>> c = Curve.from_zero_rates(tenors=[1.0, 5.0], rates=[0.03, 0.03])
>>> c.discount_factor(1.0)
0.970873...
>>> c.discount_factor([1.0, 2.0])
[0.970873..., 0.942595...]
```

For list-column projection data use the `pl.Expr` path — the Rust kernel evaluates all tenors in one pass, with no Python-level loop:

```pycon
>>> import polars as pl
>>> from gaspatchio import ActuarialFrame
>>> from gaspatchio.curves import Curve
>>> c2 = Curve.from_zero_rates(tenors=[1, 5, 10], rates=[0.01, 0.02, 0.03])
>>> af = ActuarialFrame(pl.DataFrame({"t": [[1.0, 5.0, 10.0]]}))
>>> af.df = c2.discount_factor(af["t"])
>>> af.collect()["df"].to_list()[0]
[0.990..., 0.905..., 0.7...]
```

### `fit_smith_wilson(*, tenors, rates, ufr=0.033, llp=None, alpha=None, day_count=None)`

Fit a classic Solvency II Smith-Wilson curve to zero-coupon market rates.

Solves the linear system `W @ zeta = m - mu` (see :mod:`~gaspatchio.curves._smith_wilson`) for the Wilson weights `zeta` and stores the result as a `ParametricPayload`. Subsequent evaluation via :meth:`spot_rate` dispatches to either the Rust kernel (for `pl.Expr` / list-column inputs) or the Python closed form (for scalar / array inputs), both using the same precomputed `(u, zeta, omega, alpha)`.

The `omega = log(1 + ufr)` is computed once here and carried in the payload to guarantee that the value used during the solve and the value used during evaluation are identical.

Near-duplicate tenors within 1/12 year (~1 month) of the previously- kept tenor are dropped (first of the pair wins) after sorting.

When to use: the standard EIOPA-mandated Solvency II extrapolation method for EUR, GBP, and other major currencies where the risk-free term structure must be extended beyond the Last Liquid Point (LLP) toward the Ultimate Forward Rate (UFR). Pass your liquid market zero rates (up to and including the LLP) and let alpha auto-calibrate to the EIOPA convergence criterion. The 2026 EIOPA FSP/LLFR alternative extrapolation is a planned future method (see roadmap).

Parameters:

| Name        | Type          | Description                                                                                                                 | Default                                                                                                                                                                                                                                           |
| ----------- | ------------- | --------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `tenors`    | `list[float]` | Tenor knot points in years. Must be > 0 and contain at least 1 unique tenor after de-duplication.                           | *required*                                                                                                                                                                                                                                        |
| `rates`     | `list[float]` | Annually-compounded zero rates at each tenor. Same length as tenors.                                                        | *required*                                                                                                                                                                                                                                        |
| `ufr`       | `float`       | Ultimate forward rate (annual, e.g. 0.04 for 4 %). Must satisfy ufr > -1. Defaults to 0.033 (EIOPA 2026 long-term average). | `0.033`                                                                                                                                                                                                                                           |
| `llp`       | \`float       | None\`                                                                                                                      | Last Liquid Point in years. Used as the anchor for the EIOPA convergence-point CP = max(llp + 40, 60) when alpha=None. Defaults to max(tenors) when None.                                                                                         |
| `alpha`     | \`float       | None\`                                                                                                                      | Mean-reversion speed. Must be >= 0.05. If None, alpha is calibrated automatically using the EIOPA convergence criterion: smallest alpha in [0.05, 1.0] such that the instantaneous forward rate at the convergence point is within 1 bp of omega. |
| `day_count` | \`DayCount    | None\`                                                                                                                      | Day-count convention; defaults to ActualActualISDA. Recorded for identity / source_sha only — it does not affect rate evaluation.                                                                                                                 |

Returns:

| Type    | Description                                                 |
| ------- | ----------------------------------------------------------- |
| `Curve` | A frozen :class:Curve with parametric Smith-Wilson dispatch |
| `Curve` | enabled. The tenors and rates fields hold the               |
| `Curve` | de-duplicated source observations for provenance.           |

Raises:

| Type         | Description                                                        |
| ------------ | ------------------------------------------------------------------ |
| `ValueError` | If ufr \<= -1, alpha < 0.05, or tenors and rates differ in length. |

Examples:

```pycon
>>> from gaspatchio.curves import Curve
>>> sw = Curve.fit_smith_wilson(
...     tenors=[1, 2, 3, 5, 7, 10, 15, 20],
...     rates=[0.031, 0.033, 0.034, 0.036, 0.038, 0.040, 0.041, 0.042],
... )
>>> sw.spot_rate(20.0)
0.042...
>>> sw.spot_rate(60.0)
0.037...
```

### `fit_svensson(*, tenors, rates, day_count=None)`

Fit an NSS curve to observed annually-compounded zero rates.

Uses separable nonlinear least squares (inner OLS over betas for each candidate tau pair, scored by residual SSE) to recover NSS parameters from market data. The fit is performed in continuously-compounded space (linear in the betas), so annual rates are converted to CC before fitting and the stored params are CC params consistent with :func:`~gaspatchio.curves._svensson.svensson_spot_cc`.

The source `tenors` and `rates` (annual inputs) are stored on the curve for provenance so that :meth:`canonical_form` / :meth:`source_sha` reflect the actual fitted data. Evaluation always dispatches through the NSS parametric payload, not the stored knots.

When to use: when you have a set of observed zero rates from market data (e.g. treasury strips, swap zero rates, or bootstrapped par-rate data) and want a smooth parametric curve rather than a piecewise interpolation. Requires at least 6 observations to identify all 6 NSS parameters. For curves where you already have official published NSS parameters (e.g. central-bank fitted curves), use :meth:`from_svensson` directly.

Parameters:

| Name        | Type          | Description                                                          | Default                                                                                                                           |
| ----------- | ------------- | -------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
| `tenors`    | `list[float]` | Tenor knot points in years. Must have >= 6 elements.                 | *required*                                                                                                                        |
| `rates`     | `list[float]` | Annually-compounded zero rates at each tenor. Same length as tenors. | *required*                                                                                                                        |
| `day_count` | \`DayCount    | None\`                                                               | Day-count convention; defaults to ActualActualISDA. Recorded for identity / source_sha only — it does not affect rate evaluation. |

Returns:

| Type    | Description                                                 |
| ------- | ----------------------------------------------------------- |
| `Curve` | A frozen :class:Curve with parametric dispatch enabled. The |
| `Curve` | curve's tenors and rates fields hold the source annual      |
| `Curve` | observations for provenance; parametric holds the fitted CC |
| `Curve` | NSS parameters.                                             |

Raises:

| Type         | Description                                                                      |
| ------------ | -------------------------------------------------------------------------------- |
| `ValueError` | If tenors and rates differ in length, or fewer than 6 observations are supplied. |

Examples:

```pycon
>>> from gaspatchio.curves import Curve
>>> nss = Curve.fit_svensson(
...     tenors=[1, 2, 5, 10, 20, 30],
...     rates=[0.030, 0.032, 0.035, 0.038, 0.040, 0.041],
... )
>>> nss.parametric is not None
True
>>> nss.spot_rate(10.0)
0.037...
```

### `forward_rate(*, t1, t2)`

Annually compounded forward rate between `t1` and `t2`.

Derived from the discount factors: `DF(t1) / DF(t2) = (1 + F(t1, t2))^(t2 - t1)`

Parameters:

| Name | Type    | Description                                          | Default    |
| ---- | ------- | ---------------------------------------------------- | ---------- |
| `t1` | `float` | Start year fraction. Must be strictly less than t2.  | *required* |
| `t2` | `float` | End year fraction. Must be strictly greater than t1. | *required* |

Returns:

| Type    | Description                                             |
| ------- | ------------------------------------------------------- |
| `float` | The annually compounded forward rate as a scalar float. |

Raises:

| Type         | Description  |
| ------------ | ------------ |
| `ValueError` | If t1 >= t2. |

Examples:

```pycon
>>> c = Curve.from_zero_rates(tenors=[1.0, 30.0], rates=[0.04, 0.04])
>>> c.forward_rate(t1=2.0, t2=5.0)
0.04...
```

### `from_par_rates(*, tenors, par_rates, day_count=None, interpolation='linear', extrapolation='flat')`

Build a Curve via bootstrap from annual par coupon rates.

Currently supports integer-year tenors starting at year 1 only, contiguous. Returns a Curve whose `rates` are zero rates derived via the bootstrap recursion.

Parameters:

| Name            | Type                  | Description                                                           | Default                                                                                                                           |
| --------------- | --------------------- | --------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
| `tenors`        | `list[float]`         | Integer-year tenors starting at 1, contiguous (e.g. [1.0, 2.0, 3.0]). | *required*                                                                                                                        |
| `par_rates`     | `list[float]`         | Par coupon rates at each tenor.                                       | *required*                                                                                                                        |
| `day_count`     | \`DayCount            | None\`                                                                | Day-count convention; defaults to ActualActualISDA. Recorded for identity / source_sha only — it does not affect rate evaluation. |
| `interpolation` | `InterpolationMethod` | Interpolation method; 'linear' (default) or 'log_linear'.             | `'linear'`                                                                                                                        |
| `extrapolation` | `str`                 | Behaviour outside the knot range; see :meth:from_zero_rates.          | `'flat'`                                                                                                                          |

Returns:

| Type    | Description                                                    |
| ------- | -------------------------------------------------------------- |
| `Curve` | A frozen :class:Curve whose rates are bootstrapped zero rates. |

Raises:

| Type         | Description                                                                                                              |
| ------------ | ------------------------------------------------------------------------------------------------------------------------ |
| `ValueError` | If tenors are not contiguous annual integers starting at 1, or if the underlying :meth:from_zero_rates validation fails. |

Examples:

```pycon
>>> c = Curve.from_par_rates(
...     tenors=[1.0, 2.0, 3.0], par_rates=[0.04, 0.04, 0.04]
... )
>>> c.rates
(0.04..., 0.04..., 0.04...)
```

### `from_svensson(*, b0, b1, b2, b3, tau1, tau2, day_count=None)`

Build a Curve from Nelson-Siegel-Svensson (NSS) parameters.

Implements GSW eq. 22. The curve is closed-form and does not require knot points — the parametric model is evaluated directly at any tenor.

When to use: when you have published Nelson-Siegel-Svensson parameters (e.g. from the US Federal Reserve GSW model, ECB, or central bank yield-curve publication) and want to build a smooth closed-form curve without supplying individual knot rates. The curve evaluates the NSS formula directly at any tenor — no interpolation is performed and no knot boundary is encountered, so extrapolation to very long tenors (50+ years) is well-behaved.

Parameters:

| Name        | Type       | Description                                                                                                                                                                                                                          | Default                                                                                                                           |
| ----------- | ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------- |
| `b0`        | `float`    | Level parameter (long-run continuously-compounded rate). Must satisfy b0 > 0 for a positive long-run rate in most regimes (a warning is emitted if not, but not raised — negative rate regimes are valid in ZIRP/NIRP environments). | *required*                                                                                                                        |
| `b1`        | `float`    | Slope parameter. b0 + b1 is the short-rate limit.                                                                                                                                                                                    | *required*                                                                                                                        |
| `b2`        | `float`    | First curvature parameter.                                                                                                                                                                                                           | *required*                                                                                                                        |
| `b3`        | `float`    | Second curvature parameter.                                                                                                                                                                                                          | *required*                                                                                                                        |
| `tau1`      | `float`    | First decay factor in years. Must be strictly positive.                                                                                                                                                                              | *required*                                                                                                                        |
| `tau2`      | `float`    | Second decay factor in years. Must be strictly positive.                                                                                                                                                                             | *required*                                                                                                                        |
| `day_count` | \`DayCount | None\`                                                                                                                                                                                                                               | Day-count convention; defaults to ActualActualISDA. Recorded for identity / source_sha only — it does not affect rate evaluation. |

Returns:

| Type    | Description                                             |
| ------- | ------------------------------------------------------- |
| `Curve` | A frozen :class:Curve with parametric dispatch enabled. |

Raises:

| Type         | Description                  |
| ------------ | ---------------------------- |
| `ValueError` | If tau1 \<= 0 or tau2 \<= 0. |

Examples:

```pycon
>>> from gaspatchio.curves import Curve
>>> nss = Curve.from_svensson(
...     b0=0.040, b1=-0.010, b2=0.005, b3=0.002, tau1=1.5, tau2=10.0
... )
>>> nss.parametric is not None
True
>>> nss.parametric.kind
'svensson'
>>> nss.spot_rate(7.5)
0.0402...
>>> nss.spot_rate(50)
0.0410...
```

### `from_zero_rates(*, tenors, rates, day_count=None, interpolation='linear', extrapolation='flat')`

Build a Curve from zero (spot) rates indexed by tenor in years.

The standard entry point for knot-based discount curves. Supply market zero rates at a set of liquid tenor points and choose an interpolation method; the curve fills in rates at any intermediate tenor on demand.

When to use: whenever you have a published zero-rate curve (e.g. a government bond spot curve or swap zero curve) and need to discount projected cashflows at each projection step. For parametric curves from central-bank model outputs, use :meth:`from_svensson` instead.

`tenors` and `rates` must have the same length, with `tenors` strictly increasing and at least two knots present.

Parameters:

| Name            | Type                  | Description                                                                                                                                                                                                                                           | Default                                                                                                                           |
| --------------- | --------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
| `tenors`        | `list[float]`         | Tenor knot points in years, strictly increasing.                                                                                                                                                                                                      | *required*                                                                                                                        |
| `rates`         | `list[float]`         | Annually-compounded zero rates at each knot point, same length as tenors.                                                                                                                                                                             | *required*                                                                                                                        |
| `day_count`     | \`DayCount            | None\`                                                                                                                                                                                                                                                | Day-count convention; defaults to ActualActualISDA. Recorded for identity / source_sha only — it does not affect rate evaluation. |
| `interpolation` | `InterpolationMethod` | Interpolation method; 'linear' (default), 'log_linear' (linear in log-discount-factor space, better for preserving positivity of discount factors), or 'pchip' (shape-preserving cubic Hermite, smoother forward rates).                              | `'linear'`                                                                                                                        |
| `extrapolation` | `str`                 | Behaviour outside the knot range. 'flat' (default) holds the boundary knot's spot rate; 'forward' (log_linear only) holds the last segment's forward rate — the market-consistent choice for discounting cashflows well beyond the last liquid tenor. | `'flat'`                                                                                                                          |

Returns:

| Type    | Description                     |
| ------- | ------------------------------- |
| `Curve` | A frozen :class:Curve instance. |

Raises:

| Type         | Description                                                                                                                                                     |
| ------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `ValueError` | If tenors and rates differ in length, fewer than 2 knots are supplied, tenors are not strictly increasing, or an unsupported interpolation method is requested. |

Examples:

```pycon
>>> from gaspatchio.curves import Curve
>>> c = Curve.from_zero_rates(
...     tenors=[1.0, 5.0, 10.0], rates=[0.03, 0.03, 0.03]
... )
>>> c.tenors
(1.0, 5.0, 10.0)
>>> c.interpolation
'linear'
```

PCHIP interpolation produces smoother forward rates between knots:

```pycon
>>> c_pchip = Curve.from_zero_rates(
...     tenors=[1, 2, 5, 10],
...     rates=[0.01, 0.02, 0.03, 0.035],
...     interpolation="pchip",
... )
>>> c_pchip.spot_rate(3.5)
0.0266...
>>> c_pchip.discount_factor([1.0, 5.0])
[0.990..., 0.862...]
```

### `key_rate_shift(*, tenor, bps)`

Return a new Curve with the rate at the given knot tenor shifted by `bps`.

Parameters:

| Name    | Type    | Description                                                                                                  | Default    |
| ------- | ------- | ------------------------------------------------------------------------------------------------------------ | ---------- |
| `tenor` | `float` | The knot tenor (in years) at which to apply the shift. Must be an exact member of the curve's tenors.        | *required* |
| `bps`   | `float` | Basis points to add to the single knot rate. One basis point is 0.0001 (i.e. 100 bps == 1 percentage point). | *required* |

Returns:

| Type    | Description                                                  |
| ------- | ------------------------------------------------------------ |
| `Curve` | A new frozen :class:Curve with all rates identical except at |
| `Curve` | tenor, which is incremented by bps / 10_000.                 |

Raises:

| Type         | Description                                  |
| ------------ | -------------------------------------------- |
| `ValueError` | If tenor is not an exact knot in this curve. |

Examples:

```pycon
>>> c = Curve.from_zero_rates(
...     tenors=[1.0, 5.0, 10.0], rates=[0.03, 0.04, 0.05]
... )
>>> bumped = c.key_rate_shift(tenor=5.0, bps=25)
>>> bumped.rates
(0.03, 0.0425, 0.05)
>>> c.key_rate_shift(tenor=10.0, bps=0) == c
True
```

### `shift_parallel(*, bps)`

Return a new Curve with every knot rate shifted by `bps` basis points.

Parameters:

| Name  | Type    | Description                                                                                             | Default    |
| ----- | ------- | ------------------------------------------------------------------------------------------------------- | ---------- |
| `bps` | `float` | Basis points to add to every knot rate. One basis point is 0.0001 (i.e. 100 bps == 1 percentage point). | *required* |

Returns:

| Type    | Description                                                    |
| ------- | -------------------------------------------------------------- |
| `Curve` | A new frozen :class:Curve with the same tenors, day-count, and |
| `Curve` | interpolation method, but every knot rate incremented by       |
| `Curve` | bps / 10_000.                                                  |

Examples:

```pycon
>>> c = Curve.from_zero_rates(
...     tenors=[1.0, 5.0, 10.0], rates=[0.03, 0.04, 0.05]
... )
>>> up = c.shift_parallel(bps=100)
>>> up.rates
(0.04, 0.05, 0.06...)
>>> c.shift_parallel(bps=0) == c
True
```

### `source_sha()`

Return `sha256:<hex>` over the canonical form bytes.

The digest is computed over :meth:`canonical_form` serialised by :func:`gaspatchio._identity.canonical_bytes` (sorted keys, no extra whitespace). Identical curves produce identical SHAs; any knot, day-count, or interpolation difference changes the SHA.

Returns:

| Type  | Description                                  |
| ----- | -------------------------------------------- |
| `str` | A string of the form sha256:\<64-hex-chars>. |

Examples:

```pycon
>>> a = Curve.from_zero_rates(tenors=[1.0, 5.0], rates=[0.03, 0.04])
>>> b = Curve.from_zero_rates(tenors=[1.0, 5.0], rates=[0.03, 0.04])
>>> a.source_sha() == b.source_sha()
True
>>> a.source_sha().startswith("sha256:")
True
```

### `spot_rate(t)`

Spot zero rate at year fraction(s) `t`.

Dispatches on the concrete type of `t` and returns a matching shape: scalar in → scalar out, list in → list out, ndarray in → ndarray out, Series in → Series out, Expr in → Expr out.

Supported domain is `t > 0` for `log_linear` and `smith_wilson` (the spot rate `P(t)^(-1/t) - 1` is undefined at `t = 0`); for those methods an out-of-domain `t <= 0` yields `NaN`. For any method, a non-finite `t` (NaN or ±inf) yields `NaN`. The same `NaN` sentinel is returned identically across every path and container (scalar / list / ndarray / Series / Expr).

Parameters:

| Name | Type        | Description                                                                                                              | Default    |
| ---- | ----------- | ------------------------------------------------------------------------------------------------------------------------ | ---------- |
| `t`  | `TimeInput` | Year fraction(s) at which to evaluate the spot rate. Accepts float, int, list[float], np.ndarray, pl.Series, or pl.Expr. | *required* |

Returns:

| Type    | Description |
| ------- | ----------- |
| \`float | list[float] |

Raises:

| Type        | Description                             |
| ----------- | --------------------------------------- |
| `TypeError` | If t is not one of the supported types. |

Examples:

```pycon
>>> from gaspatchio.curves import Curve
>>> c = Curve.from_zero_rates(
...     tenors=[1.0, 5.0, 10.0], rates=[0.03, 0.03, 0.03]
... )
>>> c.spot_rate(1.0)
0.03
>>> c.spot_rate([1.0, 5.0])
[0.03, 0.03]
```

For list-column projection data use the `pl.Expr` path — the Rust kernel evaluates all tenors in one pass, with no Python-level loop:

```pycon
>>> import polars as pl
>>> from gaspatchio import ActuarialFrame
>>> from gaspatchio.curves import Curve
>>> c2 = Curve.from_zero_rates(tenors=[1, 5, 10], rates=[0.01, 0.02, 0.03])
>>> af = ActuarialFrame(pl.DataFrame({"t": [[1.0, 5.0, 10.0]]}))
>>> af.r = c2.spot_rate(af["t"])
>>> af.collect()["r"].to_list()[0]
[0.01..., 0.02..., 0.03...]
```

# Date API

## Frame-Level Operations

## `gaspatchio.accessors.date.DateFrameAccessor`

Bases: `BaseFrameAccessor`

Provides date-related methods applicable to the entire ActuarialFrame.

Accessed via `.date` on an ActuarialFrame instance, e.g., `af.date`.

This accessor allows for complex date manipulations at the frame level, such as generating timelines for projections or adding durations to multiple date columns simultaneously. It integrates with Polars expressions for optimized performance.

### `add_duration(date_col, duration_str, new_col_name=None)`

Adds a duration string (e.g., '1Y', '3M', '-7d') to a date column.

This function leverages Polars' powerful duration arithmetic to efficiently modify dates within the ActuarialFrame. It can create a new column with the resulting dates or modify an existing column if `new_col_name` is not provided and `date_col` is a string name.

When to use

- **Date Arithmetic:** Use this method to shift dates by a fixed duration, such as calculating a policy anniversary, determining a future maturity date, or finding a past event date. It's particularly useful for batch operations on an entire column of dates.

Parameters:

| Name           | Type             | Description                                                    | Default                                                                                                                    |
| -------------- | ---------------- | -------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- |
| `date_col`     | `IntoExprColumn` | The column containing the dates to add the duration to.        | *required*                                                                                                                 |
| `duration_str` | `str`            | The duration string in Polars format (e.g., "1Y6M", "-3d12h"). | *required*                                                                                                                 |
| `new_col_name` | \`str            | None\`                                                         | The name for the new column containing the resulting dates. If None, modifies the original column (if it's a string name). |

Returns:

| Type             | Description                                          |
| ---------------- | ---------------------------------------------------- |
| `ActuarialFrame` | A new ActuarialFrame with the added/modified column. |

Raises:

| Type           | Description                                                                                                                |
| -------------- | -------------------------------------------------------------------------------------------------------------------------- |
| `ValueError`   | If date_col is not a valid column/expression or if modification is attempted without providing a string name for date_col. |
| `ComputeError` | If the duration addition fails (e.g., invalid duration string, incompatible date types).                                   |

Examples:

```python
import datetime
from gaspatchio import ActuarialFrame

data = {
    "event_date": [datetime.date(2023, 1, 15), datetime.date(2023, 6, 30)],
    "term_months": [6, 12],
}
af = ActuarialFrame(data)

af_plus_1y = af.date.add_duration(
    af.event_date, "1y", new_col_name="event_plus_1y"
)

print(af_plus_1y.collect())
```

```text
shape: (2, 3)
┌────────────┬─────────────┬───────────────┐
│ event_date ┆ term_months ┆ event_plus_1y │
│ ---        ┆ ---         ┆ ---           │
│ date       ┆ i64         ┆ date          │
╞════════════╪═════════════╪═══════════════╡
│ 2023-01-15 ┆ 6           ┆ 2024-01-15    │
│ 2023-06-30 ┆ 12          ┆ 2024-06-30    │
└────────────┴─────────────┴───────────────┘
```

```python
import datetime
from gaspatchio import ActuarialFrame

data = {
    "event_date": [datetime.date(2023, 1, 15), datetime.date(2023, 6, 30)],
    "term_months": [6, 12]
}
af = ActuarialFrame(data)

af_minus_3m = af.date.add_duration(af.event_date, "-3mo", new_col_name="event_minus_3m")

print(af_minus_3m.collect())
```

```text
shape: (2, 3)
┌────────────┬─────────────┬────────────────┐
│ event_date ┆ term_months ┆ event_minus_3m │
│ ---        ┆ ---         ┆ ---            │
│ date       ┆ i64         ┆ date           │
╞════════════╪═════════════╪════════════════╡
│ 2023-01-15 ┆ 6           ┆ 2022-10-15     │
│ 2023-06-30 ┆ 12          ┆ 2023-03-30     │
└────────────┴─────────────┴────────────────┘
```

### `create_timeline(start_col, end_col, freq='1d', new_col_name='timeline_date', closed='left')`

Creates timeline columns based on start and end dates.

Generates a list of dates for each row based on its start and end date, using the specified frequency. The result is exploded to create a longer DataFrame where each original row is repeated for each date in its timeline.

When to use

- **Period-to-Event Transformation:** This method is useful when you need to transform row-per-period data (where each row has a start and end date) into row-per-event data (where each row represents a specific point in time, like a month-end). For example, to calculate monthly exposures from policy start/end dates.

Parameters:

| Name           | Type             | Description                                                                                       | Default           |
| -------------- | ---------------- | ------------------------------------------------------------------------------------------------- | ----------------- |
| `start_col`    | `IntoExprColumn` | Column or expression for the start date of the interval.                                          | *required*        |
| `end_col`      | `IntoExprColumn` | Column or expression for the end date of the interval.                                            | *required*        |
| `freq`         | `str`            | The frequency of the timeline (e.g., "1M", "1Y", "1d"). Passed to pl.date_ranges.                 | `'1d'`            |
| `new_col_name` | `str`            | Name for the new column containing the generated timeline dates. Defaults to "timeline_date".     | `'timeline_date'` |
| `closed`       | `str`            | Which side of the interval is closed ("left", "right", "both", "none"). Passed to pl.date_ranges. | `'left'`          |

Returns:

| Type             | Description                                                   |
| ---------------- | ------------------------------------------------------------- |
| `ActuarialFrame` | A new ActuarialFrame instance with the original data expanded |
| `ActuarialFrame` | by the generated timeline dates.                              |

Raises:

| Type                  | Description                                                                   |
| --------------------- | ----------------------------------------------------------------------------- |
| `ColumnNotFoundError` | If start_col or end_col cannot be resolved.                                   |
| `ComputeError`        | If date range generation fails (e.g., invalid freq, incompatible date types). |

Examples:

```python
import datetime
from gaspatchio import ActuarialFrame

data = {
    "policy_id": [1, 2],
    "start_date": [datetime.date(2023, 1, 1), datetime.date(2023, 2, 15)],
    "end_date": [datetime.date(2023, 3, 1), datetime.date(2023, 4, 15)],
}
af = ActuarialFrame(data)

timeline_af = af.date.create_timeline(
    af.start_date, af.end_date, freq="1mo", new_col_name="month_end"
)

print(timeline_af.collect())
```

```text
shape: (4, 4)
┌───────────┬────────────┬────────────┬────────────┐
│ policy_id ┆ start_date ┆ end_date   ┆ month_end  │
│ ---       ┆ ---        ┆ ---        ┆ ---        │
│ i64       ┆ date       ┆ date       ┆ date       │
╞═══════════╪════════════╪════════════╪════════════╡
│ 1         ┆ 2023-01-01 ┆ 2023-03-01 ┆ 2023-01-01 │
│ 1         ┆ 2023-01-01 ┆ 2023-03-01 ┆ 2023-02-01 │
│ 2         ┆ 2023-02-15 ┆ 2023-04-15 ┆ 2023-02-15 │
│ 2         ┆ 2023-02-15 ┆ 2023-04-15 ┆ 2023-03-15 │
└───────────┴────────────┴────────────┴────────────┘
```

## Column-Level Operations

## `gaspatchio.accessors.date.DateColumnAccessor`

Bases: `BaseColumnAccessor`

Provides date-related methods for `ColumnProxy` or `ExpressionProxy` objects.

Accessed via `.date` on a column or expression, e.g., `af["my_date_col"].date`.

This accessor offers convenient methods to manipulate and extract information from date/datetime columns within Polars expressions.

### `months_between(other)`

Calculate the number of whole months between two dates.

Computes `(year2 - year1) * 12 + (month2 - month1)` where `self` is the start date and `other` is the end date. Returns a positive integer when `other` is after `self`.

This is the standard actuarial duration calculation used for policy duration in months, time-to-maturity, and assumption table key derivation.

When to use

- **Policy Duration:** Calculate months since issue for use as an assumption lookup key (mortality select period, surrender charge schedule, commission clawback period).
- **Time to Maturity:** Compute remaining term in months for each policy to determine the projection horizon or the `in_boundary` mask for IFRS 17 contract boundary.
- **Cohort Assignment:** Derive issue quarter or issue year-month for grouping policies into measurement cohorts.

##### Parameters

other : ColumnProxy | ExpressionProxy | datetime.date The end date. Can be a column reference (per-policy valuation dates), an expression, or a fixed `datetime.date` literal (single valuation date for the entire portfolio).

##### Returns

ExpressionProxy Integer number of whole months between the dates.

##### Examples

**Duration from issue date to a fixed valuation date**

```python
import datetime
from gaspatchio import ActuarialFrame

af = ActuarialFrame(
    {
        "policy_id": ["P001", "P002", "P003"],
        "issue_date": [
            datetime.date(2020, 3, 15),
            datetime.date(2018, 11, 1),
            datetime.date(2023, 7, 20),
        ],
    }
)

af.duration_months = af.issue_date.date.months_between(
    datetime.date(2025, 1, 1)
)

print(af.collect())
```

```text
shape: (3, 3)
┌───────────┬────────────┬─────────────────┐
│ policy_id ┆ issue_date ┆ duration_months │
│ ---       ┆ ---        ┆ ---             │
│ str       ┆ date       ┆ i32             │
╞═══════════╪════════════╪═════════════════╡
│ P001      ┆ 2020-03-15 ┆ 58              │
│ P002      ┆ 2018-11-01 ┆ 74              │
│ P003      ┆ 2023-07-20 ┆ 18              │
└───────────┴────────────┴─────────────────┘
```

##### Notes

- Counts whole calendar months, ignoring the day component. A policy issued on March 31 and valued on April 1 gives 1 month.
- Negative values indicate `other` is before `self`.
- For sub-monthly precision, use `date.year_frac()` instead.

##### See Also

to_period : Truncate dates to period boundaries (month, quarter, year)

### `to_period(freq='M')`

Converts a date/datetime column to a period representation (e.g., year-month).

This is useful for grouping or aggregating data by specific time periods like month, quarter, or year. It truncates the date to the beginning of the specified period.

When to use

- **Period Aggregation:** Use this to aggregate daily or weekly data into monthly, quarterly, or annual summaries.
- **Time Series Features:** For creating features for time series models based on periods.
- **Date Alignment:** When you need to align dates to a common period start (e.g., all dates in January 2023 become 2023-01-01 if `freq="M"`).

Parameters:

| Name   | Type  | Description                                                                                                                                                                                                                                                                                                  | Default |
| ------ | ----- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------- |
| `freq` | `str` | The frequency string for period conversion (e.g., "M", "Q", "Y"). See Polars documentation for truncate for available frequencies. Commonly: "1mo" (month), "1q" (quarter), "1y" (year). Note: "M", "Q", "Y" are often aliases in Polars but prefer explicit "1mo", "1q", "1y" for clarity with dt.truncate. | `'M'`   |

Returns:

| Type              | Description                                                      |
| ----------------- | ---------------------------------------------------------------- |
| `ExpressionProxy` | An ExpressionProxy representing the date column truncated to the |
| `ExpressionProxy` | specified period.                                                |

Examples:

```python
import datetime
import polars as pl
from gaspatchio import ActuarialFrame

data = {
    "event_timestamp": [
        datetime.datetime(2023, 1, 15, 10, 30, 0),
        datetime.datetime(2023, 1, 20, 14, 0, 0),
        datetime.datetime(2023, 2, 5, 8, 0, 0),
    ]
}
af = ActuarialFrame(data)

af.month = af.event_timestamp.dt.truncate("1mo").cast(pl.Date)

print(af.collect())
```

```text
shape: (3, 2)
┌─────────────────────┬────────────┐
│ event_timestamp     ┆ month      │
│ ---                 ┆ ---        │
│ datetime[μs]        ┆ date       │
╞═════════════════════╪════════════╡
│ 2023-01-15 10:30:00 ┆ 2023-01-01 │
│ 2023-01-20 14:00:00 ┆ 2023-01-01 │
│ 2023-02-05 08:00:00 ┆ 2023-02-01 │
└─────────────────────┴────────────┘
```

```python
import datetime
import polars as pl
from gaspatchio import ActuarialFrame

data = {
    "event_timestamp": [
        datetime.datetime(2023, 1, 15, 10, 30, 0),
        datetime.datetime(2023, 1, 20, 14, 0, 0),
        datetime.datetime(2023, 2, 5, 8, 0, 0),
    ]
}
af = ActuarialFrame(data)

af.year = af.event_timestamp.dt.truncate("1y").cast(pl.Date)

print(af.collect())
```

```text
shape: (3, 2)
┌─────────────────────┬────────────┐
│ event_timestamp     ┆ year       │
│ ---                 ┆ ---        │
│ datetime[μs]        ┆ date       │
╞═════════════════════╪════════════╡
│ 2023-01-15 10:30:00 ┆ 2023-01-01 │
│ 2023-01-20 14:00:00 ┆ 2023-01-01 │
│ 2023-02-05 08:00:00 ┆ 2023-01-01 │
└─────────────────────┴────────────┘
```

## Datetime Namespace

For Polars-native datetime operations (year, month, day extraction), use the `.dt` namespace directly:

## `gaspatchio.column.namespaces.dt_proxy.DtNamespaceProxy`

A proxy for Polars datetime (dt) namespace operations.

Enables type-hinting and IDE intellisense for `ActuarialFrame` datetime manipulations.

This proxy intercepts calls to datetime methods, retrieves the underlying Polars expression from its parent proxy (either a `ColumnProxy` or `ExpressionProxy`), applies the datetime operation, and then wraps the resulting Polars expression back into an `ExpressionProxy`.

### `day()`

Extract the day number of the month (1-31) from a date/datetime expression.

This function isolates the day component from a date or datetime, returning it as an integer (e.g., 15 for the 15th of the month). It works for both individual dates and lists of dates.

When to use

Extracting the day of the month can be useful in actuarial contexts for:

- **Specific Date Checks:** Identifying events occurring on particular days (e.g., end-of-month processing).
- **Intra-month Analysis:** Analyzing patterns within a month, though less common than month or year analysis.
- **Data Validation:** Ensuring dates fall within expected day ranges for specific calculations.

##### Examples

Scalar example::

```python
import polars as pl
from gaspatchio import ActuarialFrame

af = ActuarialFrame(
    {"d": pl.Series(["2023-06-05", "2023-06-15"]).str.to_date()}
)
print(af.select(af.d.dt.day().alias("day")).collect())
```

```text
shape: (2, 1)
┌─────┐
│ day │
│ --- │
│ i8  │
╞═════╡
│ 5   │
│ 15  │
└─────┘
```

Vector (list) example - loss-event days::

```python
import datetime
import polars as pl
from gaspatchio import ActuarialFrame
data = {
    "policy_id": ["E005", "F006"],
    "loss_event_dates": [
        [datetime.date(2023, 6, 5), datetime.date(2023, 6, 15)],
        [datetime.date(2024, 2, 1), datetime.date(2024, 2, 29)],
    ],
}
af = ActuarialFrame(data).with_columns(
    pl.col("loss_event_dates").cast(pl.List(pl.Date))
)
days_expr = af.loss_event_dates.dt.day()
print(af.select("policy_id", days_expr.alias("event_days")).collect())
```

```text
shape: (2, 2)
┌───────────┬────────────┐
│ literal   ┆ event_days │
│ ---       ┆ ---        │
│ str       ┆ list[i8]   │
╞═══════════╪════════════╡
│ policy_id ┆ [5, 15]    │
│ policy_id ┆ [1, 29]    │
└───────────┴────────────┘
```

### `month()`

Extract the month number (1-12) from a date or datetime expression.

This function allows you to isolate the month component from a series of dates or datetimes. The result is an integer representing the month, where January is 1 and December is 12.

When to use

In actuarial modeling, extracting the month from dates is crucial for various analyses. For instance, you might use this to:

- Analyze seasonality in claims (e.g., identifying if certain types of claims are more frequent in specific months).
- Group policies by their issue month for cohort analysis or to study underwriting patterns.
- Determine premium due dates or benefit payment schedules that occur on a monthly basis.
- Calculate fractional year components for financial calculations.

##### Examples

Scalar example::

```python
import polars as pl
from gaspatchio import ActuarialFrame

af = ActuarialFrame(
    {
        "d": pl.Series(["2022-01-01", "2022-02-01", "2022-03-01"]).str.to_date(
            "%Y-%m-%d"
        )
    }
)
print(af.select(af.d.dt.month().alias("m")).collect())
```

```text
shape: (3, 1)
┌─────┐
│ m   │
│ --- │
│ i8  │
╞═════╡
│ 1   │
│ 2   │
│ 3   │
└─────┘
```

Vector (list) example - claim-lodgement months::

```python
import datetime
import polars as pl
from gaspatchio import ActuarialFrame
data = {
    "policy_id": ["C003", "D004"],
    "claim_lodgement_dates": [
        [datetime.date(2022, 3, 10), datetime.date(2022, 4, 5)],
        [datetime.date(2023, 1, 20), datetime.date(2023, 11, 30)],
    ],
}
af = ActuarialFrame(data).with_columns(
    pl.col("claim_lodgement_dates").cast(pl.List(pl.Date))
)
months_expr = af.claim_lodgement_dates.dt.month()
result = af.select(
    pl.col("policy_id"), months_expr.alias("lodgement_months")
)
print(result.collect())
```

```text
shape: (2, 2)
┌───────────┬──────────────────┐
│ policy_id ┆ lodgement_months │
│ ---       ┆ ---              │
│ str       ┆ list[i8]         │
╞═══════════╪══════════════════╡
│ C003      ┆ [3, 4]           │
│ D004      ┆ [1, 11]          │
└───────────┴──────────────────┘
```

### `year()`

Extract the year from the underlying datetime expression.

This function isolates the year component from a date or datetime, returning it as an integer (e.g., 2023). It is applicable to both single date values and lists of dates within your `ActuarialFrame`.

When to use

Extracting the year is fundamental in actuarial analysis for:

- **Valuation and Reporting:** Determining the calendar year for financial reporting or regulatory submissions.
- **Experience Studies:** Grouping data by calendar year of event (e.g., year of claim, year of lapse) to analyze trends.
- **Cohort Analysis:** Defining cohorts based on the year of policy issue or birth year.
- **Projection Models:** Calculating durations or projecting cash flows based on calendar years.

##### Examples

Scalar example (single-date column)::

```python
import polars as pl
from gaspatchio import ActuarialFrame

data = {
    "dates": pl.Series(["2020-01-15", "2021-07-20"]).str.to_date(
        format="%Y-%m-%d"
    )
}
af = ActuarialFrame(data)
year_expr = af.dates.dt.year()
print(af.select(year_expr.alias("year")).collect())
```

```text
shape: (2, 1)
┌──────┐
│ year │
│ ---  │
│ i32  │
╞══════╡
│ 2020 │
│ 2021 │
└──────┘
```

Vector example (list-of-dates per policy)::

```python
import datetime
import polars as pl
from gaspatchio import ActuarialFrame
data_vec = {
    "policy_id": ["A001", "B002"],
    "policy_event_dates": [
        [datetime.date(2019, 12, 1), datetime.date(2020, 1, 20)],
        [
            datetime.date(2021, 5, 10),
            datetime.date(2021, 8, 15),
            datetime.date(2022, 2, 25),
        ],
    ],
}
af_vec = ActuarialFrame(data_vec)
af_vec = af_vec.with_columns(
    pl.col("policy_event_dates").cast(pl.List(pl.Date))
)
years_expr = af_vec.policy_event_dates.dt.year()
result = af_vec.select(
    pl.col("policy_id"), years_expr.alias("event_years")
)
print(result.collect())
```

```text
shape: (2, 2)
┌───────────┬────────────────────┐
│ policy_id ┆ event_years        │
│ ---       ┆ ---                │
│ str       ┆ list[i32]          │
╞═══════════╪════════════════════╡
│ A001      ┆ [2019, 2020]       │
│ B002      ┆ [2021, 2021, 2022] │
└───────────┴────────────────────┘
```

# Errors API

## PerformanceWarning

Bases: `Warning`

Warning for potential performance issues.

## `gaspatchio.accessors.excel.ExcelColumnAccessor`

Bases: `BaseColumnAccessor`

Provides Excel-related methods applicable to columns or expressions.

Accessed via `.excel` on an ActuarialFrame column or expression proxy, e.g., `af["my_excel_col"].excel`.

### `days(start_date)`

Calculate the number of days between two dates, similar to Excel's DAYS.

This function computes the number of days between an end date (the column/expression this accessor is on) and a start date. The result is positive if the end date is after the start date, and negative if before.

When to use

- **Duration Calculations**: Calculate the length of policy terms, claim periods, or other time-based intervals.
- **Age Calculations**: Determine the number of days between birth dates and valuation dates for precise age calculations.
- **Interest Calculations**: Calculate the exact number of days for interest accrual between two specific dates.
- **Exposure Period Analysis**: Measure exposure periods in days for risk assessment or premium calculations.

##### Parameters

start_date : IntoExprColumn An expression or column representing the start dates. Can be a scalar date, a column of dates.

##### Returns

ExpressionProxy An expression representing the calculated days difference as an `Int64`.

##### Examples

```python
import datetime
from gaspatchio import ActuarialFrame

data = {
    "policy_id": ["P001", "P002"],
    "start_date": [datetime.date(2023, 1, 1), datetime.date(2023, 6, 15)],
    "end_date": [datetime.date(2023, 1, 31), datetime.date(2023, 7, 15)],
}
af = ActuarialFrame(data)

# Calculate days between start and end dates
af_with_days = af.with_columns(
    days_diff=af["end_date"].excel.days(af["start_date"])
)
print(af_with_days.collect())
```

### `edate(months)`

Add months to a date, similar to Excel's EDATE.

This function adds the specified number of months to the date column/expression this accessor is on, returning the resulting date. Handles month boundaries correctly (e.g., January 31 + 1 month = February 28/29).

When to use

- **Policy Anniversary Dates**: Calculate policy renewal dates or anniversary dates by adding months to the issue date.
- **Payment Schedules**: Determine future premium due dates or benefit payment dates based on monthly intervals.
- **Maturity Calculations**: Calculate policy or investment maturity dates by adding a term in months to the start date.
- **Projection Periods**: Generate future valuation dates for cash flow projections or reserving calculations.

##### Parameters

months : IntoExprColumn An expression or column representing the number of months to add. Can be positive or negative.

##### Returns

ExpressionProxy An expression representing the date after adding months.

##### Examples

```python
import datetime
from gaspatchio import ActuarialFrame

data = {
    "policy_id": ["P001", "P002"],
    "start_date": [datetime.date(2023, 1, 31), datetime.date(2023, 3, 15)],
    "months_to_add": [1, 3],
}
af = ActuarialFrame(data)

# Add months to dates
af_with_new_dates = af.with_columns(
    new_date=af["start_date"].excel.edate(af["months_to_add"])
)
print(af_with_new_dates.collect())
```

### `eomonth(months)`

Get the end of month after adding months, similar to Excel's EOMONTH.

This function adds the specified number of months to the date column/expression this accessor is on, then returns the last day of that resulting month.

When to use

- **Reporting Periods**: Determine month-end dates for financial reporting or regulatory submissions.
- **Interest Calculations**: Calculate interest accrual periods that end on the last day of each month.
- **Benefit Payment Dates**: Set benefit payment dates to month-end when payments are made monthly.
- **Policy Term Boundaries**: Define policy terms or coverage periods that end on month boundaries.

##### Parameters

months : IntoExprColumn An expression or column representing the number of months to add. Can be positive or negative.

##### Returns

ExpressionProxy An expression representing the end-of-month date after adding months.

##### Examples

```python
import datetime
from gaspatchio import ActuarialFrame

data = {
    "policy_id": ["P001", "P002"],
    "start_date": [datetime.date(2023, 3, 15), datetime.date(2023, 1, 5)],
    "months_to_add": [1, 2],
}
af = ActuarialFrame(data)

# Get end of month after adding months
af_with_eom_dates = af.with_columns(
    end_of_month=af["start_date"].excel.eomonth(af["months_to_add"])
)
print(af_with_eom_dates.collect())
```

### `from_excel_serial(epoch='1900')`

Converts Excel serial numbers (integers or floats) to Polars Date.

Follows logic similar to openpyxl for compatibility. This method handles Excel's date serialization system, including the notorious Excel 1900 leap year bug where Excel incorrectly treats 1900 as a leap year.

When to use

- **Excel File Import:** When importing Excel files that contain date columns stored as serial numbers rather than proper date values.
- **Legacy Data Processing:** When working with older Excel files or systems that export dates as numeric serial values.
- **Cross-Platform Compatibility:** When handling Excel files that may have been created on different platforms (Windows vs Mac) with different epoch systems.
- **Data Validation:** When you need to convert and validate date serial numbers from external Excel-based data sources.

Parameters:

| Name    | Type  | Description                                                                                                                                                                                                                                                                                                                                                                                                                                                                           | Default  |
| ------- | ----- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------- |
| `epoch` | `str` | The epoch system used by Excel ('1900' or '1904'). Defaults to '1900'. 1900 Epoch (WINDOWS_1900_EPOCH = 1899-12-30): Serial 1 is 1900-01-01. Excel's serial 60 (phantom 1900-02-29) is mapped to 1900-02-28 (same as serial 59). Serials 1-59 add serial+1 days to epoch; serials > 60 add serial days to epoch. 1904 Epoch (MAC_1904_EPOCH = 1904-01-01): Serial 0 is invalid (represents time only). Serial 1 is 1904-01-02. Days to add from epoch are equal to the serial number. | `'1900'` |

Returns:

| Type              | Description                                                |
| ----------------- | ---------------------------------------------------------- |
| `ExpressionProxy` | An ExpressionProxy representing the converted date column. |

Raises:

| Type         | Description                      |
| ------------ | -------------------------------- |
| `ValueError` | If an invalid epoch is provided. |

Examples:

```python
from gaspatchio import ActuarialFrame

data = {
    "policy_id": ["P001", "P002", "P003"],
    "excel_date_serial": [44197, 44562, 44927],
}
af = ActuarialFrame(data)

af.actual_date = af.excel_date_serial.excel.from_excel_serial(epoch="1900")

print(af.collect())
```

```text
shape: (3, 3)
┌───────────┬───────────────────┬─────────────┐
│ policy_id ┆ excel_date_serial ┆ actual_date │
│ ---       ┆ ---               ┆ ---         │
│ str       ┆ i64               ┆ date        │
╞═══════════╪═══════════════════╪═════════════╡
│ P001      ┆ 44197             ┆ 2021-01-01  │
│ P002      ┆ 44562             ┆ 2022-01-01  │
│ P003      ┆ 44927             ┆ 2023-01-01  │
└───────────┴───────────────────┴─────────────┘
```

### `irr(*, guess=None, default_guess=None)`

Calculate the internal rate of return for a series of cash flows.

This function computes the discount rate that makes the net present value (NPV) of all cash flows equal to zero, using Excel's IRR algorithm.

When to use

- **Investment Analysis**: Evaluate the profitability of investment portfolios or individual securities
- **Project Evaluation**: Compare the returns of different actuarial projects or initiatives
- **Premium Adequacy**: Assess whether premium cash flows generate sufficient returns
- **Asset-Liability Matching**: Evaluate the performance of matched asset and liability cash flows

##### Parameters

guess : IntoExprColumn, optional Optional per-row initial guess for IRR. If not provided, uses default_guess. default_guess : float, optional Scalar fallback guess when `guess` is not provided. Defaults to 0.1 (10%).

##### Returns

ExpressionProxy Float64 IRR per row representing the internal rate of return.

##### Examples

Calculate IRR for investment cash flows::

````text
```python
from gaspatchio import ActuarialFrame

data = {
    "investment_id": ["INV001", "INV002"],
    "cash_flows": [
        [-1000.0, 300.0, 400.0, 500.0],  # Initial investment + returns
        [-5000.0, 1000.0, 2000.0, 3500.0]  # Different investment
    ]
}
af = ActuarialFrame(data)

af.irr = af.cash_flows.excel.irr()

print(af.collect())
````

```
shape: (2, 3)
┌───────────────┬─────────────────────────────┬──────────┐
│ investment_id ┆ cash_flows                  ┆ irr      │
│ ---           ┆ ---                         ┆ ---      │
│ str           ┆ list[f64]                   ┆ f64      │
╞═══════════════╪═════════════════════════════╪══════════╡
│ INV001        ┆ [-1000.0, 300.0, … 500.0]   ┆ 0.088963 │
│ INV002        ┆ [-5000.0, 1000.0, … 3500.0] ┆ 0.117921 │
└───────────────┴─────────────────────────────┴──────────┘
```

````

### `pv(nper, pmt, *, fv=None, typ=None)`

Calculate the present value of an investment based on periodic payments.

This function computes the present value of a loan or an investment, based on a
constant interest rate and regular payments, using Excel's PV formula.

When to use

- **Reserve Calculations**: Calculate the present value of future benefit payments for reserve valuations
- **Annuity Pricing**: Determine the present value of annuity payment streams
- **Loan Analysis**: Evaluate the present value of loan repayments for asset-liability management
- **Capital Budgeting**: Assess the present value of project cash flows for investment decisions

##### Parameters

nper : IntoExprColumn
Number of periods as scalar/column or list column.
pmt : IntoExprColumn
Payment per period as scalar/column or list column.
fv : float, optional
Future value at the end of nper periods. Defaults to 0.0.
typ : int, optional
Payment timing: 0 for payments at end of period (default), 1 for beginning.

##### Returns

ExpressionProxy
Float64 or List[Float64] representing the present value.

##### Examples

Calculate present value of annuity payments::

```text
```python
from gaspatchio import ActuarialFrame

data = {
    "policy_id": ["POL001", "POL002", "POL003"],
    "interest_rate": [0.05, 0.04, 0.06],  # Annual interest rates
    "num_periods": [10.0, 15.0, 20.0],  # Number of payment periods
    "payment": [1000.0, 1500.0, 2000.0],  # Payment per period
}
af = ActuarialFrame(data)

af.present_value = af.interest_rate.excel.pv(nper=af.num_periods, pmt=af.payment)

print(af.collect())
````

```
shape: (3, 5)
┌───────────┬───────────────┬─────────────┬─────────┬───────────────┐
│ policy_id ┆ interest_rate ┆ num_periods ┆ payment ┆ present_value │
│ ---       ┆ ---           ┆ ---         ┆ ---     ┆ ---           │
│ str       ┆ f64           ┆ f64         ┆ f64     ┆ f64           │
╞═══════════╪═══════════════╪═════════════╪═════════╪═══════════════╡
│ POL001    ┆ 0.05          ┆ 10.0        ┆ 1000.0  ┆ -7721.734929  │
│ POL002    ┆ 0.04          ┆ 15.0        ┆ 1500.0  ┆ -16677.581148 │
│ POL003    ┆ 0.06          ┆ 20.0        ┆ 2000.0  ┆ -22939.842437 │
└───────────┴───────────────┴─────────────┴─────────┴───────────────┘
```

````

### `round(num_digits=0)`

Round half away from zero, exactly like Excel's ROUND.

Excel sends a tie away from zero — `ROUND(0.125, 2)` is `0.13` —
while Polars' native `.round()` uses banker's rounding (half to
even) and gives `0.12`. When a workbook rounds inside its recursion
the tie-breaking rule compounds, so a conversion must use the
workbook's rule. The rollforward's `rf["state"].round()` op applies
this same away-from-zero convention to a running state; this method
is its column-side counterpart.

Works on scalar and per-period list columns; list columns are rounded
element-wise.

When to use

- **Workbook Conversion**: Reproduce a spreadsheet's ROUND exactly.
- **Regulatory Reporting**: Match figures rounded away from zero.
- **Reconciliation**: Remove rounding-convention mismatches first.

##### Parameters

num_digits : int
Digits after the decimal point, as in Excel: `2` rounds to
cents, `0` to whole units, negative values round to the left of
the decimal point (`-2` rounds to hundreds). Defaults to 0.

##### Returns

ExpressionProxy
Float64 (or List[Float64] for list columns) rounded half away
from zero.

##### Examples

Round per-period charges to cents the way the workbook does::

```text
```python
from gaspatchio import ActuarialFrame

af = ActuarialFrame({"coi": [[10.125, 7558.485]]})
af.coi_rounded = af.coi.excel.round(2)
````

````

### `yearfrac(end_date_expr, basis='act/act')`

Calculate the year fraction between two dates, similar to Excel's YEARFRAC.

This function computes the fraction of a year represented by the number of
whole days between a start date (the column/expression this accessor is on)
and an end date. It uses a specified day count basis.

When to use

- **Premium Proration**: Calculate the portion of an annual premium that corresponds to a partial policy term, for example, if a policy starts or ends mid-year.
- **Exposure Calculation**: Determine fractional exposure periods for reserving or IBNR (Incurred But Not Reported) calculations, especially when dealing with policies that are not in force for a full year.
- **Investment Analysis**: Compute fractional year periods for accrued interest calculations or for annualizing returns on investments held for parts of a year.
- **Performance Metrics**: Analyze time-based metrics such as time-to-claim or duration of an event, expressed as a fraction of a year.

##### Parameters

end_date_expr : IntoExprColumn
An expression or column representing the end dates. Can be a scalar date,
a column of dates.
basis : int or str, optional
The day count basis to use. Can be an integer (0-4) or a string name.
Defaults to "act/act" (which is basis 1).

```text
Supported bases:
- `0` or `'us_nasd_30_360'` (30/360 US NASD) - US (NASD) 30/360 convention
- `1` or `'act/act'` (Actual/Actual) - Actual/Actual convention
- `2` or `'actual_360'` (Actual/360) - Actual/360 convention
- `3` or `'actual_365'` (Actual/365 fixed) - Actual/365 convention
- `4` or `'european_30_360'` (30/360 European) - European 30/360 convention
````

##### Returns

ExpressionProxy An expression representing the calculated year fraction as a `Float64`.

##### Raises

TypeError If the underlying proxy for the start date is not a `ColumnProxy` or `ExpressionProxy`. RuntimeError If the operation requires an `ActuarialFrame` context that is not available. ValueError If an invalid basis is provided.

##### Examples

Calculating Policy Term as Year Fraction (Scalar/Column Operations)::

````text
Scenario: You have policy start and end dates and want to calculate the policy term in years.

```python
import datetime
from gaspatchio import ActuarialFrame

data = {
    "policy_id": ["P001", "P002", "P003"],
    "start_date": [
        datetime.date(2020, 1, 1),
        datetime.date(2021, 6, 15),
        datetime.date(2022, 3, 1),
    ],
    "end_date": [
        datetime.date(2021, 1, 1),
        datetime.date(2022, 6, 15),
        datetime.date(2022, 9, 1),
    ],
}
af = ActuarialFrame(data)

af.term_years = af.start_date.excel.yearfrac(af.end_date, basis="act/act")

print(af.collect())
````

```
shape: (3, 4)
┌───────────┬────────────┬────────────┬────────────┐
│ policy_id ┆ start_date ┆ end_date   ┆ term_years │
│ ---       ┆ ---        ┆ ---        ┆ ---        │
│ str       ┆ date       ┆ date       ┆ f64        │
╞═══════════╪════════════╪════════════╪════════════╡
│ P001      ┆ 2020-01-01 ┆ 2021-01-01 ┆ 1.0        │
│ P002      ┆ 2021-06-15 ┆ 2022-06-15 ┆ 1.0        │
│ P003      ┆ 2022-03-01 ┆ 2022-09-01 ┆ 0.50411    │
└───────────┴────────────┴────────────┴────────────┘
```

````

List Column Workaround::

```text
For actuarial projections stored as list columns (e.g., monthly projection dates),
use the explode/group_by pattern:

```python
import datetime
import polars as pl
from gaspatchio import ActuarialFrame
from gaspatchio.accessors.excel_functions.yearfrac import yearfrac

# Example with monthly projection dates
projection_data = {
    "policy_id": ["P001", "P002"],
    "projection_dates": [
        [datetime.date(2024, i, 1) for i in range(1, 13)],  # 12 monthly dates
        [datetime.date(2024, i, 15) for i in range(1, 13)]
    ],
    "maturity_date": [
        datetime.date(2024, 12, 31),
        datetime.date(2025, 1, 1)
    ]
}
af_proj = ActuarialFrame(projection_data)

# Calculate yearfrac for each projection date using explode/group_by
result = (
    af_proj.collect()
    .lazy()
    .with_row_index("_idx")
    .explode("projection_dates")
    .with_columns(
        yearfrac(pl.col("projection_dates"), pl.col("maturity_date"), basis="act/act")
        .alias("years_to_maturity")
    )
    .group_by("_idx")
    .agg([
        pl.col("policy_id").first(),
        pl.col("years_to_maturity"),
        pl.col("maturity_date").first()
    ])
    .drop("_idx")
    .collect()
)
print(result)
````

```
shape: (2, 3)
┌───────────┬──────────────────────────────────┬───────────────┐
│ policy_id ┆ years_to_maturity                ┆ maturity_date │
│ ---       ┆ ---                              ┆ ---           │
│ str       ┆ list[f64]                        ┆ date          │
╞═══════════╪══════════════════════════════════╪═══════════════╡
│ P001      ┆ [0.997268, 0.912568, … 0.081967] ┆ 2024-12-31    │
│ P002      ┆ [0.961749, 0.877049, … 0.046575] ┆ 2025-01-01    │
└───────────┴──────────────────────────────────┴───────────────┘
```

Note: List columns are not directly supported due to Polars plugin limitations. Excel 365 achieves this with dynamic arrays, but we require explicit data transformation.

```
```

# Expression API

## ColumnProxy

Represents a column identifier within an ActuarialFrame, acting as a starting point for expressions.

## `captures`

Access capture fields from a tracked rollforward.

Returns a dict-like accessor so that `af.av.captures["name"]` lazily extracts the `Capture(name)` field from the hidden rollforward Struct column.

## `date`

Access date-related column operations.

## `excel`

Access excel-related column operations.

## `finance`

Access finance-related column operations.

## `increments`

Access per-step increment fields from a tracked rollforward.

Returns a dict-like accessor so that `af.av.increments["Premium"]` lazily extracts the `Premium` increment series from the hidden rollforward Struct column.

Requires the rollforward to be built with `track_increments=True`; each labelled op then emits its signed per-period delta (negative for charges, positive for credits, zero after a stop or lapse).

## `shape`

Resolved shape of this column reference (`scalar`, `list`, or `unknown`).

## `map_batches(func, return_dtype=None)`

Apply a Python function to the entire column as a Series.

This is more efficient than apply for operations that can process multiple values at once, especially for NumPy or vector operations.

Parameters:

| Name           | Type       | Description                                                   | Default    |
| -------------- | ---------- | ------------------------------------------------------------- | ---------- |
| `func`         | `Callable` | Function that receives a Series and returns a Series or array | *required* |
| `return_dtype` |            | Optional polars DataType for the result                       | `None`     |

Returns:

| Name              | Type              | Description                     |
| ----------------- | ----------------- | ------------------------------- |
| `ExpressionProxy` | `ExpressionProxy` | Result of applying the function |

## `map_elements(func, return_dtype=None)`

Apply a Python function to each element of the column.

Parameters:

| Name           | Type       | Description                             | Default    |
| -------------- | ---------- | --------------------------------------- | ---------- |
| `func`         | `Callable` | Function to apply to each element       | *required* |
| `return_dtype` |            | Optional polars DataType for the result | `None`     |

Returns:

| Name              | Type              | Description                     |
| ----------------- | ----------------- | ------------------------------- |
| `ExpressionProxy` | `ExpressionProxy` | Result of applying the function |

## ExpressionProxy

Represents a Polars expression derived from ActuarialFrame operations or ColumnProxy methods.

## `date`

Access date-related expression operations.

## `finance`

Access finance-related expression operations.

## `kind`

Resolved kind: explicit override > dtype-driven fallback > value.

## `shape`

Resolved shape of this expression.

# Finance Operations

Finance operations are available at two levels:

- **Frame-level**: Called on `af.finance` (operates on the entire frame)
- **Column-level**: Called on `af.column.finance` (operates on a specific column)

## Frame-Level Operations

## `gaspatchio.accessors.finance.FinanceFrameAccessor`

Bases: `BaseFrameAccessor`

Financial operations at the frame level for ActuarialFrame.

Provides frame-level methods for financial calculations that require coordinated operations across multiple columns, particularly for list columns in actuarial projections.

Accessed via `.finance` on an ActuarialFrame instance.

#### Methods

discount_factor(rate_col, periods_col, output_col, method="spot") Calculate discount factors from list columns using native Polars. Supports spot and forward discounting without map_elements. present_value(cashflow_col, rate_col, period_col) Calculate present value of cash flows.

### `discount_factor(rate_col, periods_col, output_col, method='spot')`

Calculate discount factors for projection timelines using list_pow plugin.

Computes present value discount factors v^t from interest rates across entire projection timelines. Uses Rust list_pow plugin for optimal performance - eliminates EXPLODE/GROUP_BY pattern for 10x+ speedup on list columns.

When to use

- **Reserve Calculations:** Discount future benefit payments and expenses to present value for statutory reserves, GAAP liabilities, or embedded value calculations.
- **Cash Flow Projections:** Calculate present values of projected premiums, claims, and expenses in profit testing models for product pricing and profitability analysis.
- **Guaranteed Benefits:** Discount guaranteed minimum death benefits (GMDB), withdrawal benefits (GMWB), or income benefits (GMIB) in variable annuity and equity-indexed product valuations.
- **Economic Scenario Testing:** Apply forward rate curves from economic scenario generators for stochastic reserve calculations and risk capital models.

##### Parameters

rate_col : str Name of column containing interest rates (as lists). Typically monthly rates for projection periods. periods_col : str Name of column containing time periods (as lists). Must align element-wise with rate_col. output_col : str Name for the new column containing discount factors. method : {"spot", "forward"}, default "spot" Discount method:

```text
- "spot": v[t] = (1 + r)^(-t) - Same rate for all periods
- "forward": v[t] = ∏(1 + r[i])^(-1) for i < t - Period-varying rates
```

##### Returns

ActuarialFrame New frame with discount factor column added.

##### Examples

**Spot Discounting: Constant Rate**

```python
from gaspatchio import ActuarialFrame

data = {
    "policy_id": [1, 2],
    "monthly_rate": [[0.004, 0.004, 0.004], [0.003, 0.003]],
    "month": [[0, 1, 2], [0, 1]],
}
af = ActuarialFrame(data)

af = af.finance.discount_factor(
    rate_col="monthly_rate",
    periods_col="month",
    output_col="disc_factors",
    method="spot",
)

print(af.collect())
```

```text
shape: (2, 4)
┌───────────┬───────────────────────┬───────────┬───────────────────────────┐
│ policy_id ┆ monthly_rate          ┆ month     ┆ disc_factors              │
│ ---       ┆ ---                   ┆ ---       ┆ ---                       │
│ i64       ┆ list[f64]             ┆ list[i64] ┆ list[f64]                 │
╞═══════════╪═══════════════════════╪═══════════╪═══════════════════════════╡
│ 1         ┆ [0.004, 0.004, 0.004] ┆ [0, 1, 2] ┆ [1.0, 0.996016, 0.992048] │
│ 2         ┆ [0.003, 0.003]        ┆ [0, 1]    ┆ [1.0, 0.997009]           │
└───────────┴───────────────────────┴───────────┴───────────────────────────┘
```

**Forward Discounting: Varying Rates**

```python
from gaspatchio import ActuarialFrame

data = {
    "policy_id": [1],
    "forward_rates": [[0.003, 0.004, 0.005]],
    "month": [[0, 1, 2]],
}
af = ActuarialFrame(data)

af = af.finance.discount_factor(
    rate_col="forward_rates",
    periods_col="month",
    output_col="disc_factors",
    method="forward",
)

print(af.collect())
```

```text
shape: (1, 4)
┌───────────┬───────────────────────┬───────────┬───────────────────────────┐
│ policy_id ┆ forward_rates         ┆ month     ┆ disc_factors              │
│ ---       ┆ ---                   ┆ ---       ┆ ---                       │
│ i64       ┆ list[f64]             ┆ list[i64] ┆ list[f64]                 │
╞═══════════╪═══════════════════════╪═══════════╪═══════════════════════════╡
│ 1         ┆ [0.003, 0.004, 0.005] ┆ [0, 1, 2] ┆ [1.0, 0.997009, 0.993037] │
└───────────┴───────────────────────┴───────────┴───────────────────────────┘
```

##### See Also

to_monthly : Convert annual rates to monthly rates

### `present_value(cashflow_col, rate_col, period_col)`

Calculate the present value of cash flows.

Computes present value using the formula `PV = CF / (1 + rate)^period`, assuming cash flows occur at the end of each period. Essential for discounting future cash flows in pricing, reserving, and valuation.

When to use

- **Reserve Calculations:** Discount future benefit payments and expenses to calculate policy reserves under various standards.
- **Product Pricing:** Calculate present value of expected premiums and claims to determine pricing margins and profitability.
- **Embedded Value:** Discount projected profits for embedded value and value of in-force business calculations.
- **Cash Flow Testing:** Present value cash flows for asset adequacy testing and scenario analysis.

##### Parameters

cashflow_col : IntoExprColumn Column containing the cash flow amounts to discount. rate_col : IntoExprColumn Column with discount rate per period (e.g., 0.05 for 5% annual). period_col : IntoExprColumn Column with period number (1, 2, 3...). Must be >= 1.

##### Returns

ExpressionProxy Present value of each cash flow discounted to time zero.

##### Examples

**Scalar Example: Discount policy cash flows**

```python
from gaspatchio import ActuarialFrame

data = {
    "policy_id": ["P001", "P002"],
    "cashflow": [1000.0, 2000.0],
    "rate": [0.05, 0.04],
    "period": [1, 2],
}
af = ActuarialFrame(data)

af.pv = af.finance.present_value(af.cashflow, af.rate, af.period)

print(af.collect())
```

```text
shape: (2, 5)
┌───────────┬──────────┬──────┬────────┬─────────────┐
│ policy_id ┆ cashflow ┆ rate ┆ period ┆ pv          │
│ ---       ┆ ---      ┆ ---  ┆ ---    ┆ ---         │
│ str       ┆ f64      ┆ f64  ┆ i64    ┆ f64         │
╞═══════════╪══════════╪══════╪════════╪═════════════╡
│ P001      ┆ 1000.0   ┆ 0.05 ┆ 1      ┆ 952.380952  │
│ P002      ┆ 2000.0   ┆ 0.04 ┆ 2      ┆ 1849.112426 │
└───────────┴──────────┴──────┴────────┴─────────────┘
```

##### See Also

discount_factor : Calculate discount factors for projection timelines

## Column-Level Operations

## `gaspatchio.accessors.finance.FinanceColumnAccessor`

Bases: `BaseColumnAccessor`

Financial mathematics and valuation operations.

Provides methods for rate conversion and present value computations on columns or expressions.

Accessed via `.finance` on an ActuarialFrame column or expression proxy, e.g., `af["annual_rate"].finance.to_monthly()`.

#### Methods

to_monthly(method="compound") Convert annual interest rates to monthly rates discount(rate_expr, n_periods_expr) Discount values using specified rate and periods

#### Notes

For discount factor calculations on list columns, use the frame-level accessor: `af.finance.discount_factor(rate_col, periods_col, output_col)`

### `compound(rate, periods_per_year)`

Calculate compound growth factor for periods.

Computes growth factors using the formula `(1 + rate)^(period / periods_per_year)`. Commonly used for inflation adjustments, investment growth, or other compound growth calculations in actuarial projections.

When to use

- **Inflation Adjustments:** Calculate inflation factors for expenses, benefits, or premiums that grow with inflation over projection periods.
- **Investment Growth:** Model accumulation of funds under compound interest assumptions.
- **Benefit Increases:** Calculate growth factors for benefits that increase at a fixed compound rate.

##### Parameters

rate : float Annual growth rate (e.g., 0.01 for 1% annual growth) periods_per_year : int Number of periods per year (e.g., 12 for monthly, 4 for quarterly)

##### Returns

ExpressionProxy Growth factors with same structure as input column (scalar or list)

##### Examples

**List column example: Monthly inflation factors**

```python
from gaspatchio import ActuarialFrame

data = {"month": [[0, 1, 6, 12]]}
af = ActuarialFrame(data)

af.inflation_factor = af.month.finance.compound(rate=0.01, periods_per_year=12)

print(af.collect())
```

**Scalar column example: Quarterly growth**

```python
from gaspatchio import ActuarialFrame

data = {"quarter": [0, 1, 4]}
af = ActuarialFrame(data)

af.growth_factor = af.quarter.finance.compound(rate=0.02, periods_per_year=4)

print(af.collect())
```

##### See Also

to_monthly : Convert annual rates to monthly rates discount_factor : Calculate discount factors from interest rates

### `discount(rate_expr, n_periods_expr)`

Discount the value in the current column/expression.

Computes discounted values using the formula `Discounted Value = Value / (1 + rate)^n_periods`. Useful for converting future values to present values with flexible rate and period inputs from other columns or expressions.

When to use

- **Future Value Discounting:** Convert guaranteed maturity values, surrender values, or death benefits to present value.
- **Reserve Calculations:** Discount projected liabilities back to valuation date for statutory or GAAP reserves.
- **Profit Testing:** Calculate present value of future profits in pricing models and profitability analysis.
- **Investment Returns:** Discount expected investment income or fund values in unit-linked or variable product models.

##### Parameters

rate_expr : IntoExprColumn The discount rate per period (e.g., 0.05 for 5% annual). Can be a scalar, column reference, or expression. n_periods_expr : IntoExprColumn The number of periods to discount over. Can be a scalar, column reference, or expression.

##### Returns

ExpressionProxy Discounted values with same structure as input column.

##### Examples

**Scalar Example: Discount future values**

```python
from gaspatchio import ActuarialFrame

data = {
    "policy_id": ["P001", "P002"],
    "future_value": [1050.0, 2080.0],
    "rate": [0.05, 0.04],
    "periods": [1, 2],
}
af = ActuarialFrame(data)

af.present_value = af.future_value.finance.discount(af.rate, af.periods)

print(af.collect())
```

```text
shape: (2, 5)
┌───────────┬──────────────┬──────┬─────────┬───────────────┐
│ policy_id ┆ future_value ┆ rate ┆ periods ┆ present_value │
│ ---       ┆ ---          ┆ ---  ┆ ---     ┆ ---           │
│ str       ┆ f64          ┆ f64  ┆ i64     ┆ f64           │
╞═══════════╪══════════════╪══════╪═════════╪═══════════════╡
│ P001      ┆ 1050.0       ┆ 0.05 ┆ 1       ┆ 1000.0        │
│ P002      ┆ 2080.0       ┆ 0.04 ┆ 2       ┆ 1923.076923   │
└───────────┴──────────────┴──────┴─────────┴───────────────┘
```

##### See Also

present_value : Calculate present value of cash flows at frame level discount_factor : Calculate discount factors for projection timelines

### `to_monthly(method='compound')`

Convert annual interest rate to monthly rate.

Transforms annual effective interest rates to equivalent monthly rates using either compound or simple interest conventions. Essential for actuarial projections with monthly timesteps when assumptions are provided annually.

For list columns, applies conversion element-wise within each list. For scalar columns, applies conversion to each row value.

When to use

- **Monthly Projections:** Convert annual discount rates, investment returns, or interest crediting rates to monthly equivalents for cash flow models with monthly timesteps.
- **Pricing Models:** Transform annual pricing assumptions to monthly rates for variable annuity, universal life, or investment-linked product models.
- **Reserve Calculations:** Convert annual reserve discount rates to monthly for mid-month or monthly reserve valuations.
- **Policy Loans:** Calculate monthly interest accrual on policy loans when loan terms specify annual interest rates.

##### Parameters

method : {"compound", "simple"}, default "compound" Conversion method:

- "compound": (1 + r_annual)^(1/12) - 1 (standard actuarial practice)
- "simple": r_annual / 12 (linear approximation)

##### Returns

ExpressionProxy Monthly interest rate with same structure as input (scalar or list)

##### Examples

**Scalar Example: Convert annual discount rates**

```python
from gaspatchio import ActuarialFrame

data = {"annual_rate": [0.05, 0.06, 0.04]}
af = ActuarialFrame(data)

af.monthly_rate = af.annual_rate.finance.to_monthly()

print(af.collect())
```

```text
shape: (3, 2)
┌─────────────┬──────────────┐
│ annual_rate ┆ monthly_rate │
│ ---         ┆ ---          │
│ f64         ┆ f64          │
╞═════════════╪══════════════╡
│ 0.05        ┆ 0.004074     │
│ 0.06        ┆ 0.004868     │
│ 0.04        ┆ 0.003274     │
└─────────────┴──────────────┘
```

**Scalar Example: Simple conversion for approximation**

```python
from gaspatchio import ActuarialFrame

data = {"annual_rate": [0.05, 0.06, 0.04]}
af = ActuarialFrame(data)

af.monthly_rate = af.annual_rate.finance.to_monthly(method="simple")

print(af.collect())
```

```text
shape: (3, 2)
┌─────────────┬──────────────┐
│ annual_rate ┆ monthly_rate │
│ ---         ┆ ---          │
│ f64         ┆ f64          │
╞═════════════╪══════════════╡
│ 0.05        ┆ 0.004167     │
│ 0.06        ┆ 0.005        │
│ 0.04        ┆ 0.003333     │
└─────────────┴──────────────┘
```

**Vector Example: Projection timeline with varying rates**

```python
from gaspatchio import ActuarialFrame

data = {"annual_rates": [[0.05, 0.05, 0.06, 0.06]]}
af = ActuarialFrame(data)

af.monthly_rates = af.annual_rates.finance.to_monthly()

print(af.collect())
```

```text
shape: (1, 2)
┌──────────────────────┬─────────────────────────────────┐
│ annual_rates         ┆ monthly_rates                   │
│ ---                  ┆ ---                             │
│ list[f64]            ┆ list[f64]                       │
╞══════════════════════╪═════════════════════════════════╡
│ [0.05, 0.05, … 0.06] ┆ [0.004074, 0.004074, … 0.00486… │
└──────────────────────┴─────────────────────────────────┘
```

##### Notes

- Compound method is standard actuarial practice (maintains equivalence)
- Simple method provides linear approximation (less accurate but faster)
- For list columns, conversion is applied to each element
- Formula: Compound = (1 + r_annual)^(1/12) - 1, Simple = r_annual / 12

##### See Also

discount_factor : Calculate discount factors from interest rates

# API Overview

Gaspatchio's API is designed to feel like Excel formulas but work like optimized Python. Every operation is vectorized by default, planned before it runs, and built for actuarial workflows.

## The Three API Layers

### 1. Core API

The foundation for all actuarial modeling:

| Component          | Purpose                                                              |
| ------------------ | -------------------------------------------------------------------- |
| **ActuarialFrame** | Container for policy data; calculations run together on `.collect()` |
| **Expressions**    | Column operations and arithmetic                                     |
| **Conditionals**   | `when().then().otherwise()` business logic                           |
| **Scenarios**      | What-if analysis and sensitivity testing                             |

### 2. Domain Extensions (Accessors)

Specialized namespaces attached to columns for domain-specific operations:

| Accessor        | Access Pattern                | Purpose                                                       |
| --------------- | ----------------------------- | ------------------------------------------------------------- |
| **Excel**       | `af.column.excel`             | Excel-compatible functions (YEARFRAC, DAYS, PV, IRR)          |
| **Date**        | `af.column.dt`                | Date extraction and manipulation                              |
| **Finance**     | `af.column.finance`           | Rate conversions, discounting, compounding                    |
| **Projection**  | `af.column.projection`        | Time-series operations (cumulative survival, period shifting) |
| **Rollforward** | `af.projection.rollforward()` | Account value rollforward with state-dependent steps          |
| **String**      | `af.column.str`               | String manipulation                                           |

### 3. Assumption Tables

The `Table` class for high-performance dimension-based lookups. See [Assumptions](https://gaspatchio.dev/0.9.0/concepts/assumptions/index.md) for details.

## Quick Start Pattern

```python
from gaspatchio import ActuarialFrame, when
from gaspatchio.assumptions import Table
import datetime

# 1. Create frame from policy data
af = ActuarialFrame({
    "policy_id": ["P001", "P002"],
    "issue_date": [datetime.date(2020, 1, 15), datetime.date(2021, 6, 1)],
    "issue_age": [35, 42],
    "sum_assured": [100_000, 250_000],
    "status": ["active", "active"],
    "annual_rate": [0.04, 0.05],
})

# 2. Date calculations with Excel accessor
af.years_in_force = af.issue_date.excel.yearfrac(
    datetime.date(2024, 12, 31), basis="act/act"
)

# 3. Arithmetic (auto-vectorized)
af.attained_age = af.issue_age + af.years_in_force

# 4. Financial calculations
af.pv_premiums = af.annual_rate.excel.pv(nper=20, pmt=1200)

# 5. Conditional business logic
af.benefit = when(af.status == "active").then(af.sum_assured).otherwise(0)

# 6. Execute all calculations at once
df = af.collect()
```

## The Accessor Pattern

Accessors extend columns with domain-specific methods. They're available at two levels:

```python
import datetime
from gaspatchio import ActuarialFrame

# Extended frame with columns required by the accessor examples below.
af = ActuarialFrame({
    "policy_id":    ["P001",                    "P002"],
    "issue_date":   [datetime.date(2020, 1, 15), datetime.date(2021, 6, 1)],
    "val_date":     [datetime.date(2024, 12, 31), datetime.date(2024, 12, 31)],
    "issue_age":    [35,       42],
    "sum_assured":  [500_000,  250_000],
    "status":       ["active", "active"],
    "rate":         [[0.004, 0.004, 0.004], [0.003, 0.003, 0.003]],
    "qx":           [[0.00120, 0.00135, 0.00152], [0.00180, 0.00202, 0.00228]],
    "mortality_rates": [[0.00120, 0.00135, 0.00152], [0.00180, 0.00202, 0.00228]],
    "premiums":     [[500.0, 500.0, 500.0], [750.0, 750.0, 750.0]],
    "x":            [1_000.0, 2_000.0],
    "y":            [50.0,    80.0],
    "amount":       [500_000, 250_000],
    "policy class": ["term-10", "whole-life"],
})
```

**Column-level** - operates on a specific column:

```python
af.years = af.issue_date.excel.yearfrac(af.val_date, basis="act/act")
af.month = af.issue_date.dt.month()
af.survival = af.qx.projection.cumulative_survival()
```

**Frame-level** - operates on the entire frame:

```python
af = af.finance.discount_factor(
    rate_col="rate",
    periods_col="month",
    output_col="disc_factor"
)
```

## Key Concepts

### Plan First, Run Once

Each column assignment is recorded rather than executed straight away. Nothing runs until you call `.collect()`:

```python
# These just build the plan
af.a = af.x + af.y
af.b = af.a * 2
af.c = af.b.excel.pv(nper=10, pmt=100)

# This executes everything in one optimized pass
df = af.collect()
```

### Vector-Aware Operations

List columns represent time series. All operations work element-wise automatically:

```python
# Scalar * list = list (broadcast)
af.claims = af.sum_assured * af.mortality_rates  # mortality_rates is a list column

# List - list = list (element-wise)
af.net_cf = af.premiums - af.claims
```

### Attribute Notation

Always use attribute notation for clean, readable code:

```python
# Preferred
af.result = af.rate * af.amount

# Only use brackets for reserved words or special characters
af["class"] = af["policy class"].str.to_uppercase()
```

## Navigation Guide

| I want to...                              | Go to...                                                                    |
| ----------------------------------------- | --------------------------------------------------------------------------- |
| Create and manipulate frames              | [ActuarialFrame](https://gaspatchio.dev/0.9.0/api/actuarial_frame/index.md) |
| Use Excel functions (PV, IRR, YEARFRAC)   | [Excel](https://gaspatchio.dev/0.9.0/api/excel/index.md)                    |
| Work with dates                           | [Date](https://gaspatchio.dev/0.9.0/api/date/index.md)                      |
| Calculate discount factors, convert rates | [Finance](https://gaspatchio.dev/0.9.0/api/finance/index.md)                |
| Handle time-series (survival, shifting)   | [Projection](https://gaspatchio.dev/0.9.0/api/projection/index.md)          |
| Run account value rollforwards            | [Rollforward](https://gaspatchio.dev/0.9.0/api/rollforward/index.md)        |
| Write if/then logic                       | [Conditionals](https://gaspatchio.dev/0.9.0/api/conditionals/index.md)      |
| Run scenarios and sensitivities           | [Scenarios](https://gaspatchio.dev/0.9.0/api/scenarios/index.md)            |
| Look up assumption tables                 | [Assumptions](https://gaspatchio.dev/0.9.0/concepts/assumptions/index.md)   |

## Further Reading

- [Concepts Introduction](https://gaspatchio.dev/0.9.0/concepts/intro/index.md) - Deeper architectural understanding
- [Assumptions Guide](https://gaspatchio.dev/0.9.0/concepts/assumptions/index.md) - Table lookups and dimension handling
- [Scenarios Guide](https://gaspatchio.dev/0.9.0/concepts/scenarios/index.md) - What-if analysis patterns

# Mortality API

For the actuarial framing — why a `MortalityTable` records age basis and select period explicitly, the three structures (aggregate, select-ultimate, joint), automatic duration clamping — see [Mortality Tables](https://gaspatchio.dev/0.9.0/concepts/mortality/index.md).

## MortalityTable

## `gaspatchio.MortalityTable`

## Conventions

## `gaspatchio.mortality.AgeBasis = Literal['age_last_birthday', 'age_nearest_birthday']`

## `gaspatchio.mortality.Structure = Literal['aggregate', 'select_ultimate', 'joint']`

## Frame-Level Operations

## `gaspatchio.accessors.projection_frame.ProjectionFrameAccessor`

Bases: `BaseFrameAccessor`

Frame-level accessor for actuarial projection operations.

Two verbs

- `set(...)` — declare the projection time axis on the frame
- `rollforward(...)` — construct a state-machine rollforward builder that reads the projection from this frame

### `anniversary_mask()`

Return per-row List of length n_periods marking anniversaries.

### `canonical_form()`

Return the structural recipe — same shape as Schedule.canonical_form().

### `contract_boundary(*, end_date_column=None)`

Return per-row List of length n_periods — kernel termination mask.

True at period t means the contract has terminated by period t. Pass to `af.projection.rollforward(..., contract_boundary=...)` to bound the projection at each policy's end date.

This is the **negation** of :meth:`is_in_force` — the kernel uses boundary semantics (True = terminate); `is_in_force()` is natural for other uses (True = active).

### `is_in_force(*, end_date_column=None)`

Return per-row List of length n_periods — boundary mask.

Pass `end_date_column` for from_inception schedules where each policy has its own end date. Without it, the mask is uniform True for all periods.

### `period_dates()`

Return per-row List.

Uniform schedules give length `n_periods+1` for every row; `per_policy_grid` gives each policy its own (variable) length.

### `rollforward(*, states, points=None, track_increments=False, lapse_when_all_non_positive=(), contract_boundary=None, batch_axes=('policy',), schedule=None)`

Construct a :class:`RollforwardBuilder` that reads schedule from this frame.

`schedule=` is no longer accepted on this method — call `af.projection.set(...)` first. Every other keyword forwards to :class:`RollforwardBuilder` unchanged.

### `set(*, schedule=None, valuation_date=None, until=None, until_value=None, issue_age_column='issue_age', inception_column='policy_inception', start_date=None, n_periods=None, frequency=None, per_policy=None)`

Declare the projection time axis on this frame.

With a per-policy `until_value` column, each policy projects only as far as its own horizon — producing variable-length (jagged) list columns instead of one uniform grid sized to the longest-lived policy. This recovers the compute/memory of per-policy timelines while keeping the unified projection API.

`per_policy` (default `None` = auto): jagged is the default whenever it applies (a column `until_value` with a `term_*` horizon); other cases fall back to a uniform grid. Pass `per_policy=False` to force a uniform (rectangular) grid — useful when combining fixed-width list columns or doing cross-policy shared-axis aggregation. Pass `per_policy=True` to require jagged (raises if it cannot apply). `rollforward()` works on both uniform and jagged timelines.

Stamps `projection_start_date`, `projection_end_date`, `num_proj_months`, and — at month-aligned frequencies on a projection-anchored axis — `month`: elapsed whole months from the projection start, length `n_periods + 1`, aligned with `t_years()`. Derive a year label in the model with the convention you mean: `af.month // 12` (completed years / duration) or `ceil(month / 12)` ("year 1" ordinal). There is deliberately no `proj_year` or `year` column.

Raises:

| Type         | Description                                                                                                                                            |
| ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `ValueError` | if the frame already carries a month column and has no in-session projection — rename your column, or drop it if it came from a previous run's output. |

See ref/38-projection-axis/specs for full semantics.

### `source_sha()`

Return sha256: over the canonical form bytes (audit identifier).

### `t_years()`

Return per-row List of cumulative year fractions from 0.

Length is `n_periods + 1`. Feeds `Curve.discount_factor(t)` directly.

### `year_fractions()`

Return per-row List of length n_periods (per-period dt[t]).

## Column-Level Operations

## `gaspatchio.accessors.projection.ProjectionColumnAccessor`

Bases: `BaseColumnAccessor`

Actuarial projection operations for time-series calculations.

This accessor provides methods for transforming rates and probabilities into cumulative values over projection periods. Complex operations like cumulative products use these methods, while simple operations like multiplication should use standard operators.

Design Philosophy

- Complex cumulative operations: Use projection methods
- Simple arithmetic: Use operators (`*`, `+`, `-`, `/`)
- Terminal/aggregate values: Use Polars (`.list.last()`)

Accessed via `.projection` on a column or expression, e.g., `af["mortality_rate"].projection.cumulative_survival()`.

Examples:

Cumulative survival from mortality rates:

```python
from gaspatchio import ActuarialFrame

data = {"qx": [[0.001, 0.0011, 0.0012], [0.002, 0.0022, 0.0024]]}
af = ActuarialFrame(data)

# Complex cumulative product - use projection method
af.survival_to_t = af.qx.projection.cumulative_survival()

# Simple multiplication - use operators
af.death_benefit = af.face_amount * af.survival_to_t * af.qx
af.premium = af.annual_premium * af.survival_to_t

# Terminal value - use Polars
af.maturity_benefit = af.face_amount.list.last()
```

Premium holiday modeling:

```python
from gaspatchio import ActuarialFrame

data = {"premium": [[1000, 1000, 1000, 1000, 1000]]}
af = ActuarialFrame(data)

# Period override - use projection method
af.premium_with_holiday = af.premium.projection.with_period(3, value=0)
# Result: [1000, 1000, 1000, 0, 1000]
```

### `accumulate(*, initial, multiply, add)`

Accumulate values using a linear recurrence.

Computes `state[t] = state[t-1] * multiply[t] + add[t]` for each time step, returning all intermediate states as a list column. This is the core primitive for account value rollforwards and other state-dependent actuarial projections where cashflows at time *t* depend on accumulated state at *t-1*.

The actuary pre-computes the multiplicative and additive components in Python, keeping business logic readable, while the Rust kernel handles the tight sequential loop per policy. Polars parallelises across policies automatically.

When to use

- **Account Value Rollforward:** Accumulate account values where premiums, fees, and investment returns interact with the running balance each period.
- **Reserve Accumulation:** Build up statutory or GAAP reserves period-by-period using interest and cashflow assumptions.
- **Universal Life Projections:** Model COI deductions, crediting rates, and expense charges that depend on the current account value via Picard iteration with `accumulate()`.
- **Unit-Linked Fund Projection:** Project fund values forward where management charges are deducted as a proportion of the current fund value.

##### Parameters

initial : str or pl.Expr or ExpressionProxy or ColumnProxy Initial state per policy (e.g., starting account value). A scalar column with one value per row. Broadcasts when length is 1. multiply : float or str or pl.Expr or ExpressionProxy or ColumnProxy Multiplicative growth factor per time step (e.g., `1 + interest_rate`). A list column with one list per policy, or a scalar (number or scalar column) — a scalar broadcasts to the `add` list's per-policy lengths. add : float or str or pl.Expr or ExpressionProxy or ColumnProxy Additive flow per time step (e.g., premiums minus charges, grown by the interest factor). A list column with one list per policy, or a scalar — broadcasts to `multiply`'s per-policy lengths. When both are lists, inner lengths must match. At least one of `multiply`/`add` must be a list: two scalars leave no timeline to broadcast onto, and the growth-free case has a closed form (`initial + flow.cum_sum()`).

##### Returns

ExpressionProxy List column of accumulated values at each time step.

##### Examples

**Simple Account Value Rollforward**

```python
from gaspatchio import ActuarialFrame

data = {
    "av_init": [1000.0, 2000.0],
    "growth": [[1.01, 1.01, 1.01], [1.02, 1.02, 1.02]],
    "net_flow": [[50.0, 50.0, 50.0], [100.0, 100.0, 100.0]],
}
af = ActuarialFrame(data)

af.av = af.growth.projection.accumulate(
    initial="av_init",
    multiply="growth",
    add="net_flow",
)
print(af.collect()["av"].to_list())
# [[1060.0, 1120.6, 1181.8059999999998], [2140.0, 2282.8, 2428.456]]
```

**Scalar Growth Factor**

A scalar `multiply` broadcasts to the flow's per-policy timeline — each policy's list length sets its own repetition count:

```python
from gaspatchio import ActuarialFrame

af = ActuarialFrame(
    {
        "av_init": [1000.0],
        "net_flow": [[100.0, 100.0, 100.0]],
    }
)
af.av = af.net_flow.projection.accumulate(
    initial="av_init",
    multiply=1.5,
    add="net_flow",
)
print(af.collect()["av"].to_list())
# [[1600.0, 2500.0, 3850.0]]
```

### `at_period(relative_period, fill_value=0.0)`

Get value at relative period offset.

Access values from other time periods using mathematical t notation. Negative values reference prior periods (t-1, t-2), positive values reference future periods (t+1, t+2).

This method provides flexible time-shifting for arbitrary period offsets, complementing the convenience methods `previous_period()` (t-1) and `next_period()` (t+1).

For list columns, shifts values within each list. For scalar columns, shifts across rows (use `.over()` for grouping).

When to use

- **Multi-Period Lag Analysis:** Access values from multiple periods back (t-2, t-3) for trend analysis and smoothing calculations.
- **Reserve Rollforward:** Reference reserves from specific prior periods in complex reserve formulas requiring multiple lag periods.
- **Experience Studies:** Compare values across multiple time periods to analyze experience trends and validate assumptions.
- **Flexible Time-Shifting:** Use when previous_period() and next_period() don't provide the specific offset needed for your calculation.

##### Parameters

relative_period : int Period offset from current time using mathematical notation:

- Negative values: prior periods (e.g., -1 for t-1, -2 for t-2)
- Positive values: future periods (e.g., 1 for t+1, 2 for t+2)
- Zero: current period (no shift) fill_value : scalar, optional Value to use for missing entries at boundaries. Default is 0.

##### Returns

ExpressionProxy Expression with values from specified relative period

##### Examples

**Previous Period: t-1**

```python
from gaspatchio import ActuarialFrame

data = {"reserve": [[1000, 1100, 1200]]}
af = ActuarialFrame(data)

# at_period(-1) is equivalent to previous_period()
af.reserve_t1 = af.reserve.projection.at_period(-1)

print(af.collect())
```

```text
shape: (1, 2)
┌──────────────────┬──────────────────┐
│ reserve          ┆ reserve_t1       │
│ ---              ┆ ---              │
│ list[i64]        ┆ list[i64]        │
╞══════════════════╪══════════════════╡
│ [1000, 1100, ...] ┆ [0, 1000, 1100]  │
└──────────────────┴──────────────────┘
```

**Two Periods Back: t-2**

```python
from gaspatchio import ActuarialFrame

data = {"value": [[100, 110, 120, 130, 140]]}
af = ActuarialFrame(data)

af.value_t2 = af.value.projection.at_period(-2)

print(af.collect())
```

```text
shape: (1, 2)
┌───────────────────────┬──────────────────────┐
│ value                 ┆ value_t2             │
│ ---                   ┆ ---                  │
│ list[i64]             ┆ list[i64]            │
╞═══════════════════════╪══════════════════════╡
│ [100, 110, 120, 13... ┆ [0, 0, 100, 110, 120]│
└───────────────────────┴──────────────────────┘
```

**Next Period: t+1**

```python
from gaspatchio import ActuarialFrame

data = {"cashflow": [[1000, 1100, 1200]]}
af = ActuarialFrame(data)

# at_period(1) is equivalent to next_period()
af.cf_tp1 = af.cashflow.projection.at_period(1)

print(af.collect())
```

```text
shape: (1, 2)
┌──────────────────┬─────────────────┐
│ cashflow         ┆ cf_tp1          │
│ ---              ┆ ---             │
│ list[i64]        ┆ list[i64]       │
╞══════════════════╪═════════════════╡
│ [1000, 1100, ...] ┆ [1100, 1200, 0] │
└──────────────────┴─────────────────┘
```

**Reserve Rollforward Formula**

```python
from gaspatchio import ActuarialFrame

data = {
    "reserve": [[0, 950, 1900, 2850]],
    "premium": [[1000, 1000, 1000, 1000]],
    "interest": [[50, 52, 55, 58]],
    "benefit": [[100, 102, 105, 108]],
}
af = ActuarialFrame(data)

# Reserve rollforward formula:
# Reserve(t) = Reserve(t-1) + Premium(t) + Interest(t) - Benefit(t)
af.reserve_t1 = af.reserve.projection.at_period(-1)
af.reserve_calc = af.reserve_t1 + af.premium + af.interest - af.benefit

print(af.collect())
```

```text
shape: (1, 6)
┌─────────────────┬─────────────┬─────────┬─────────┬──────────┬──────────────┐
│ reserve         ┆ premium     ┆ intere.. ┆ benefit ┆ reserve..┆ reserve_calc │
│ ---             ┆ ---         ┆ ---     ┆ ---     ┆ ---      ┆ ---          │
│ list[i64]       ┆ list[i64]   ┆ list..  ┆ list..  ┆ list[i64]┆ list[i64]    │
╞═════════════════╪═════════════╪═════════╪═════════╪══════════╪══════════════╡
│ [0, 950, 19...┆ [1000, 10...┆ [50, 52...┆ [100, ...┆ [0, 0, 9...┆ [950, 19...│
└─────────────────┴─────────────┴─────────┴─────────┴──────────┴──────────────┘
```

##### See Also

previous_period : Convenience method for t-1 next_period : Convenience method for t+1

### `broadcast_to_periods(like=None)`

Broadcast a per-policy scalar column across the projection periods.

Repeats this column's value once per projection period, producing a list column aligned with the frame's axis. Works for **every dtype** — strings, booleans, and categoricals included — unlike the numeric idiom `af.scalar + af.list * 0.0`, which has no string equivalent and previously forced hand-rolled `repeat_by` helpers into model code.

When to use

- **String dimensions in lookups:** A per-policy `occupation_class` or `smoker_status` feeding a per-period `Table.lookup` needs one value per period, not one per policy.
- **Per-period conditionals on policy attributes:** Broadcasting a label before a `when()` that selects between per-period lists.
- **Any scalar-to-period alignment** where arithmetic broadcasting does not apply (non-numeric dtypes).

##### Parameters

like : ColumnProxy or ExpressionProxy or pl.Expr, optional A list column whose per-row lengths define the broadcast length — use this on frames without a projection axis, or to match a specific column's (possibly jagged) lengths. Defaults to the frame's `month` period index, stamped by `projection.set()`.

##### Returns

ExpressionProxy List column repeating this column's value once per period.

##### Raises

ValueError If `like` is not given and the frame has no `month` column to take period lengths from.

##### Examples

**String attribute broadcast to the projection axis**

```python
from gaspatchio import ActuarialFrame

data = {
    "occupation_class": ["M", "H"],
    "month": [[0, 1, 2], [0, 1, 2]],
}
af = ActuarialFrame(data)

af.occupation_per_period = af.occupation_class.projection.broadcast_to_periods()

print(af.collect())
```

```text
shape: (2, 3)
┌──────────────────┬───────────┬───────────────────────┐
│ occupation_class ┆ month     ┆ occupation_per_period │
│ ---              ┆ ---       ┆ ---                   │
│ str              ┆ list[i64] ┆ list[str]             │
╞══════════════════╪═══════════╪═══════════════════════╡
│ M                ┆ [0, 1, 2] ┆ ["M", "M", "M"]       │
│ H                ┆ [0, 1, 2] ┆ ["H", "H", "H"]       │
└──────────────────┴───────────┴───────────────────────┘
```

**Matching a specific (jagged) list column with `like=`**

```python
from gaspatchio import ActuarialFrame

data = {
    "frequency": ["monthly", "annual"],
    "premiums": [[100.0, 100.0], [1200.0, 1200.0, 1200.0]],
}
af = ActuarialFrame(data)

af.frequency_per_period = af.frequency.projection.broadcast_to_periods(
    like=af.premiums
)

print(af.collect())
```

```text
shape: (2, 3)
┌───────────┬──────────────────┬───────────────────────────┐
│ frequency ┆ premiums         ┆ frequency_per_period      │
│ ---       ┆ ---              ┆ ---                       │
│ str       ┆ list[f64]        ┆ list[str]                 │
╞═══════════╪══════════════════╪═══════════════════════════╡
│ monthly   ┆ [100.0, 100.0]   ┆ ["monthly", "monthly"]    │
│ annual    ┆ [1200.0, … 1200… ┆ ["annual", "annual", "an… │
└───────────┴──────────────────┴───────────────────────────┘
```

##### See Also

previous_period : Access prior-period values on the same axis

### `cumulative_survival(rate_timing=None, start_at=1.0, *, from_survival=False)`

Convert mortality rates to cumulative survival probabilities.

Transforms period mortality rates (qx) into cumulative survival probabilities using the formula `tpx[t] = (1-qx[0]) * (1-qx[1]) * ... * (1-qx[t])`. Essential for life insurance projections, reserve calculations, and any actuarial work requiring survival probabilities from mortality assumptions.

When the source hands you a **survival-shaped** factor instead — a combined persistency `px = (1-qx)(1-lapse)`, extremely common in reserving — pass `from_survival=True` and the factors cumulate directly, with no inversion anywhere. Spelling it as `1 - px` so the method can undo it is a lossy round trip: `1 - (1 - px)` is exact only while `px >= 0.5`, and a shock-lapse year sits well below that. The error is tiny (~1e-17) but avoidable for free, and a strict tie-out sees it.

For list columns, applies element-wise cumulative product within each list. For scalar columns, applies cumulative product across rows (use `.over()` for grouping by policy).

When to use

- **Life Insurance Projections:** Calculate the probability policies remain inforce for death benefit, premium, and cash value projections.
- **Reserve Calculations:** Compute expected policy counts for reserve valuations and capital requirements.
- **Persistency Analysis:** Model combined mortality and lapse decrements to project policy persistency over time.
- **Pricing Models:** Calculate expected present values of benefits and premiums weighted by survival probabilities.

##### Timing Conventions

The `rate_timing` parameter controls when decrement rates are applied:

- **beginning_of_period** (default): Rate at period t is NOT yet applied to PIF. The survival at t represents the probability of surviving TO the start of period t. Result: `[1.0, tpx[0], tpx[0]*tpx[1], ...]`
- **end_of_period**: Rate at period t HAS been applied to PIF. The survival at t represents the probability of surviving THROUGH period t. This matches Excel-style timing. Result: `[tpx[0], tpx[0]*tpx[1], ...]`

With constant rates, both conventions give identical values. The difference only appears when rates change over time (e.g., at age boundaries).

##### Parameters

rate_timing : {"beginning_of_period", "end_of_period"}, optional When decrement rates are applied. Recommended for most users:

```text
- ``"beginning_of_period"``: Rate at t NOT yet applied (default behavior)
- ``"end_of_period"``: Rate at t HAS been applied (Excel-style)

If not specified, falls back to `start_at` parameter behavior.
```

start_at : float, optional Lower-level control over timing. Only use if `rate_timing` is not set. Initial survival probability to prepend at t=0:

```text
- 1.0 (default): Beginning-of-period [1.0, tpx[0], tpx[1], ...]
- None: End-of-period [tpx[0], tpx[1], ...]
- Other: Custom initial value (e.g., 0.95 for partial cohort)
```

from_survival : bool, optional The column already holds survival-shaped factors (px, persistency), not decrement rates. Factors cumulate directly — no `1 - x` anywhere — so shock-lapse years below px = 0.5 stay bit-exact. Timing parameters mean exactly the same thing in both shapes.

##### Returns

ExpressionProxy Cumulative survival probabilities for each period

##### Raises

ValueError If both `rate_timing` and a non-default `start_at` are specified, or if `rate_timing` has an invalid value RuntimeError If the column is not part of an ActuarialFrame context

##### Examples

**Beginning-of-Period Timing (Default)**

```python
from gaspatchio import ActuarialFrame

data = {
    "policy_id": ["P001", "P002"],
    "qx": [[0.001, 0.002, 0.003], [0.002, 0.003, 0.004]],
}
af = ActuarialFrame(data)

# Default: rate at t not yet applied
af.pols_if = af.qx.projection.cumulative_survival()
# Or explicitly:
af.pols_if = af.qx.projection.cumulative_survival(
    rate_timing="beginning_of_period"
)

print(af.collect())
```

```text
shape: (2, 3)
┌───────────┬───────────────────────┬────────────────────────┐
│ policy_id ┆ qx                    ┆ pols_if                │
│ ---       ┆ ---                   ┆ ---                    │
│ str       ┆ list[f64]             ┆ list[f64]              │
╞═══════════╪═══════════════════════╪════════════════════════╡
│ P001      ┆ [0.001, 0.002, 0.003] ┆ [1.0, 0.999, 0.997002] │
│ P002      ┆ [0.002, 0.003, 0.004] ┆ [1.0, 0.998, 0.995006] │
└───────────┴───────────────────────┴────────────────────────┘
```

**End-of-Period Timing (Excel-Style)**

```python
from gaspatchio import ActuarialFrame

data = {
    "qx": [[0.001, 0.002, 0.003]],
}
af = ActuarialFrame(data)

# Excel-style: rate at t has been applied
af.tpx = af.qx.projection.cumulative_survival(rate_timing="end_of_period")

print(af.collect())
```

```text
shape: (1, 2)
┌───────────────────────┬─────────────────────────────┐
│ qx                    ┆ tpx                         │
│ ---                   ┆ ---                         │
│ list[f64]             ┆ list[f64]                   │
╞═══════════════════════╪═════════════════════════════╡
│ [0.001, 0.002, 0.003] ┆ [0.999, 0.997002, 0.994011] │
└───────────────────────┴─────────────────────────────┘
```

**Survival-Shaped Input (Combined Persistency)**

```python
from gaspatchio import ActuarialFrame

data = {
    "px": [[0.999, 0.998, 0.997]],
}
af = ActuarialFrame(data)

# The column already holds survival factors - cumulate them directly
af.cum_px = af.px.projection.cumulative_survival(
    rate_timing="end_of_period", from_survival=True
)

print(af.collect())
```

```text
shape: (1, 2)
┌───────────────────────┬─────────────────────────────┐
│ px                    ┆ cum_px                      │
│ ---                   ┆ ---                         │
│ list[f64]             ┆ list[f64]                   │
╞═══════════════════════╪═════════════════════════════╡
│ [0.999, 0.998, 0.997] ┆ [0.999, 0.997002, 0.994011] │
└───────────────────────┴─────────────────────────────┘
```

**Custom Initial Value (Partial Cohort)**

```python
from gaspatchio import ActuarialFrame

data = {
    "qx": [[0.001, 0.002, 0.003]],
}
af = ActuarialFrame(data)

# 95% survived underwriting - use start_at for custom values
af.pols_if = af.qx.projection.cumulative_survival(start_at=0.95)

print(af.collect())
```

```text
shape: (1, 2)
┌───────────────────────┬─────────────────────────┐
│ qx                    ┆ pols_if                 │
│ ---                   ┆ ---                     │
│ list[f64]             ┆ list[f64]               │
╞═══════════════════════╪═════════════════════════╡
│ [0.001, 0.002, 0.003] ┆ [0.95, 0.999, 0.997002] │
└───────────────────────┴─────────────────────────┘
```

### `next_period(fill_value=0.0)`

Get value from next period (t+1).

Equivalent to shifting forward one period. Less common than `previous_period()` but useful for certain actuarial calculations requiring forward-looking values.

For list columns, shifts values within each list. For scalar columns, shifts across rows (use `.over()` for grouping).

When to use

- **Forward-Looking Calculations:** Access next period values for calculations that require looking ahead in the projection timeline.
- **Period-Over-Period Growth:** Calculate growth rates or changes by comparing current values to next period values.
- **Validation Checks:** Verify that projected values follow expected patterns by comparing current and next period results.
- **Timing Adjustments:** Reference future period values when modeling payment or benefit timing that leads the valuation period.

##### Parameters

fill_value : scalar, optional Value to use for last period where no next value exists. Default is 0.

##### Returns

ExpressionProxy Expression with values shifted from next period

##### Examples

**Basic Usage: Next Period Values**

```python
from gaspatchio import ActuarialFrame

data = {"interest_rate": [[0.05, 0.06, 0.07]]}
af = ActuarialFrame(data)

af.rate_next = af.interest_rate.projection.next_period()

print(af.collect())
```

```text
shape: (1, 2)
┌────────────────────┬───────────────────┐
│ interest_rate      ┆ rate_next         │
│ ---                ┆ ---               │
│ list[f64]          ┆ list[f64]         │
╞════════════════════╪═══════════════════╡
│ [0.05, 0.06, 0.07] ┆ [0.06, 0.07, 0.0] │
└────────────────────┴───────────────────┘
```

**Forward-Looking Calculation Example**

```python
from gaspatchio import ActuarialFrame

data = {"cashflow": [[1000, 1100, 1200]]}
af = ActuarialFrame(data)

# Compare current period to next period
af.cf_next = af.cashflow.projection.next_period()
af.cf_growth = af.cf_next - af.cashflow

print(af.collect())
```

```text
shape: (1, 3)
┌──────────────────┬─────────────────┬──────────────────┐
│ cashflow         ┆ cf_next         ┆ cf_growth        │
│ ---              ┆ ---             ┆ ---              │
│ list[i64]        ┆ list[i64]       ┆ list[i64]        │
╞══════════════════╪═════════════════╪══════════════════╡
│ [1000, 1100, ...] ┆ [1100, 1200, 0] ┆ [100, 100, -...] │
└──────────────────┴─────────────────┴──────────────────┘
```

##### See Also

previous_period : Get value from previous period (t-1) at_period : Get value at arbitrary period offset

### `previous_period(fill_value=0.0)`

Get value from previous period (t-1).

Equivalent to shifting back one period. Most common case for actuarial projections when referencing prior period values.

For list columns, shifts values within each list. For scalar columns, shifts across rows (use `.over()` for grouping).

When to use

- **Inforce Rollforward:** Calculate beginning-of-period inforce values using ending inforce from the previous period in life insurance models.
- **Reserve Calculations:** Access prior period reserves for reserve rollforward formulas and cash flow testing.
- **Period Comparisons:** Compare current period values against previous period for variance analysis and experience studies.
- **Dependent Calculations:** Reference lagged values in formulas where current period depends on prior period results.

##### Parameters

fill_value : scalar, optional Value to use for first period where no previous value exists. Default is 0.

##### Returns

ExpressionProxy Expression with values shifted from previous period

##### Examples

**Basic Usage: Previous Period Values**

```python
from gaspatchio import ActuarialFrame

data = {"pols_death": [[10, 15, 20]]}
af = ActuarialFrame(data)

af.pols_death_prev = af.pols_death.projection.previous_period()

print(af.collect())
```

```text
shape: (1, 2)
┌──────────────┬──────────────────┐
│ pols_death   ┆ pols_death_prev  │
│ ---          ┆ ---              │
│ list[i64]    ┆ list[i64]        │
╞══════════════╪══════════════════╡
│ [10, 15, 20] ┆ [0, 10, 15]      │
└──────────────┴──────────────────┘
```

**Custom Fill Value: Reserve Calculations**

```python
from gaspatchio import ActuarialFrame

data = {"reserve": [[1000, 1100, 1200]]}
af = ActuarialFrame(data)

# Use None to get null for missing values
af.reserve_prev = af.reserve.projection.previous_period(fill_value=None)

print(af.collect())
```

```text
shape: (1, 2)
┌──────────────────┬──────────────────┐
│ reserve          ┆ reserve_prev     │
│ ---              ┆ ---              │
│ list[i64]        ┆ list[i64]        │
╞══════════════════╪══════════════════╡
│ [1000, 1100, ...] ┆ [null, 1000, ...] │
└──────────────────┴──────────────────┘
```

**Actuarial Formula: Inforce Rollforward**

```python
from gaspatchio import ActuarialFrame

data = {
    "pols_if_after_death": [[1000, 990, 975]],
    "pols_lapse": [[5, 8, 10]],
}
af = ActuarialFrame(data)

# Calculate beginning-of-period inforce using previous period values
# pols_if_bop(t) = pols_if_after_death(t-1) - pols_lapse(t-1)
af.pols_if_prev = af.pols_if_after_death.projection.previous_period(
    fill_value=1000
)
af.pols_lapse_prev = af.pols_lapse.projection.previous_period()
af.pols_if_bop = af.pols_if_prev - af.pols_lapse_prev

print(af.collect())
```

```text
shape: (1, 4)
┌─────────────────────┬─────────────┬────────────────┬─────────────┐
│ pols_if_after_death ┆ pols_lapse  ┆ pols_if_prev   ┆ pols_if_bop │
│ ---                 ┆ ---         ┆ ---            ┆ ---         │
│ list[i64]           ┆ list[i64]   ┆ list[i64]      ┆ list[i64]   │
╞═════════════════════╪═════════════╪════════════════╪═════════════╡
│ [1000, 990, 975]    ┆ [5, 8, 10]  ┆ [1000, 1000... ┆ [1000, 995..│
└─────────────────────┴─────────────┴────────────────┴─────────────┘
```

##### See Also

next_period : Get value from next period (t+1) at_period : Get value at arbitrary period offset

### `prospective_value(discount_rate=None, discount_factor=None, *, timing='end_of_period')`

Calculate prospective (present) value of future cashflows from each time t.

Computes the present value of all future cashflows from each projection period onwards, using backward recursion: PV(t) = CF(t) + PV(t+1) * v(t).

This is the standard actuarial "prospective policy value" calculation, essential for reserve valuations, embedded value projections, profit testing, and asset adequacy testing. Replaces complex Polars list operations with a clean, actuarial-focused API.

When to use

- **Reserve Calculations:** Compute present value of future benefits less premiums for statutory and GAAP reserve valuations.
- **Embedded Value:** Calculate present value of future profits for embedded value and value of in-force business metrics.
- **Profit Testing:** Project present value of cashflows at each duration for pricing validation and profitability analysis.
- **Asset Adequacy:** Test sufficiency of assets to cover future liabilities under various interest rate scenarios.

##### Parameters

discount_rate : float or ExpressionProxy or ColumnProxy, optional Per-period discount rate for discounting future cashflows:

```text
- Scalar float: Constant rate for all periods (e.g., 0.05 for 5%)
- List column: Per-period rates that may vary over time

Cannot be specified together with `discount_factor`.
```

discount_factor : ExpressionProxy or ColumnProxy, optional Pre-computed cumulative discount factors. Use when you have yield curve or scenario-specific discount factors already calculated.

```text
**Convention (timing-dependent):** ``discount_factor[t]`` must
discount the position-``t`` cashflow *at the instant implied by
the timing* back to the valuation point:

- ``timing="end_of_period"``: the position-0 cashflow is paid one
  period out, so ``discount_factor[0]`` already includes one
  period's discounting — ``[v, v*v, v*v*v, ...]``, **not 1.0
  first**.
- ``timing="beginning_of_period"``: the position-0 cashflow is
  immediate — ``[1.0, v, v*v, ...]``.

Supplying beginning-anchored factors (leading 1.0) with
``end_of_period`` timing shifts every period's discounting by one
period and leaves the first cashflow undiscounted.

Cannot be specified together with `discount_rate`.
```

timing : {"beginning_of_period", "end_of_period"}, default "end_of_period" When cashflows occur within each period, following the Excel PV/annuity convention:

```text
- ``"end_of_period"`` (Excel ``type=0``, ordinary annuity): the cashflow at
  ``t`` is paid at the end of the period, so every cashflow is discounted a
  full period. This is the SMALLER value. Typical for benefits.
- ``"beginning_of_period"`` (Excel ``type=1``, annuity-due): the cashflow at
  ``t`` is paid at the start of the period, so the first cashflow is
  undiscounted. This is the LARGER value. Typical for premiums.
```

##### Returns

ExpressionProxy Present value of future cashflows at each projection period

##### Raises

ValueError If both `discount_rate` and `discount_factor` are specified, or if neither is specified

##### Examples

**Death Benefit PV with Constant Discount Rate**

```python
from gaspatchio import ActuarialFrame

data = {
    "death_benefit": [[100.0, 100.0, 100.0]],
}
af = ActuarialFrame(data)

# Calculate prospective value at 5% discount rate.
# Default timing is "end_of_period" (Excel type=0, ordinary annuity):
# every cashflow is discounted a full period, e.g. at t=0
# 100/1.05 + 100/1.05^2 + 100/1.05^3 = 272.32.
af.pv_benefits = af.death_benefit.projection.prospective_value(
    discount_rate=0.05
)

print(af.collect())
```

```text
shape: (1, 2)
┌────────────────────┬─────────────────────────────┐
│ death_benefit      ┆ pv_benefits                 │
│ ---                ┆ ---                         │
│ list[f64]          ┆ list[f64]                   │
╞════════════════════╪═════════════════════════════╡
│ [100.0, 100.0, ... ┆ [272.32, 185.94, 95.24]     │
└────────────────────┴─────────────────────────────┘
```

**Premium PV with Time-Varying Rates**

```python
from gaspatchio import ActuarialFrame

data = {
    "premium": [[1000.0, 1000.0, 1000.0]],
    "disc_rate": [[0.04, 0.05, 0.06]],
}
af = ActuarialFrame(data)

af.pv_premiums = af.premium.projection.prospective_value(
    discount_rate=af.disc_rate,
    timing="beginning_of_period"
)

print(af.collect())
```

**With Pre-Computed Discount Factors**

```python
from gaspatchio import ActuarialFrame

data = {
    "benefit": [[100.0, 100.0, 100.0]],
    # 5% end-of-period factors: position 0 pays one period out, so the
    # first factor is v = 1/1.05, not 1.0 (see discount_factor docs).
    "v_t": [[0.952381, 0.907029, 0.863838]],
}
af = ActuarialFrame(data)

af.pv = af.benefit.projection.prospective_value(discount_factor=af.v_t)

print(af.collect())
```

##### Notes

**Implementation Details:**

The method internally performs:

1. Compute discounted cashflows: CF(t) * v(t)
1. Apply reverse -> cumsum -> reverse pattern to get "sum from t to end"
1. Adjust for timing convention

NaN cashflows propagate into every affected period's PV — a NaN means an upstream defect (e.g. a lookup miss under `on_missing="nan"`). Zero beyond-term periods explicitly with `when/otherwise`.

**Replaces Ugly Pattern:**

This method replaces verbose Polars list manipulation. The old pattern required 6+ lines of Polars list operations (reverse, cumsum, reverse), while the new API is a single clean method call.

##### See Also

cumulative_survival : Calculate cumulative survival probabilities previous_period : Access prior period values for reserve rollforward

### `remaining_sum()`

Compute backward cumulative sum (remaining sum from each period to end).

For each element at position `t`, returns the sum of all elements from `t` to the end of the list. Equivalent to reversing the list, computing a cumulative sum, then reversing back.

This is the standard pattern for IFRS 17 coverage unit remaining totals, remaining premium factors, and annuity-due calculations where you need "sum of all future values from this point."

When to use

- **IFRS 17 Coverage Units:** Compute `cu_remaining[t]` — the total remaining coverage from period `t` onward — for CSM amortisation factor calculation: `amort[t] = cu[t] / cu_remaining[t]`.
- **Remaining Premium Factors:** Calculate the remaining expected premium stream from each period for net premium reserve formulas.
- **Annuity-Due Factors:** Derive the running sum of future discount factors for annuity-due present value calculations.

##### Returns

ExpressionProxy List column where each element is the sum from that position to the end of the original list.

##### Examples

**IFRS 17 coverage unit remaining totals**

```python
from gaspatchio import ActuarialFrame

af = ActuarialFrame({
    "policy_id": ["P001", "P002"],
    "coverage_units": [
        [100, 90, 80, 70, 60],
        [200, 180, 160, 140, 120],
    ],
})

af.cu_remaining = af.coverage_units.projection.remaining_sum()

print(af.collect().select(["policy_id", "coverage_units", "cu_remaining"]))
```

```text
shape: (2, 3)
┌───────────┬───────────────────┬───────────────────┐
│ policy_id ┆ coverage_units    ┆ cu_remaining      │
│ ---       ┆ ---               ┆ ---               │
│ str       ┆ list[i64]         ┆ list[i64]         │
╞═══════════╪═══════════════════╪═══════════════════╡
│ P001      ┆ [100, 90, … 60]   ┆ [400, 300, … 60]  │
│ P002      ┆ [200, 180, … 120] ┆ [800, 600, … 120] │
└───────────┴───────────────────┴───────────────────┘
```

##### Notes

- At the last period, `remaining_sum` equals the element itself.
- At `t=0`, it equals the total sum of the entire list.
- The relationship `amort_factor[t] = cu[t] / cu_remaining[t]` gives the proportion of CSM to release in each period.

##### See Also

cumulative_survival : Forward cumulative product for survival probabilities prospective_value : Present value of future cashflows from each period

### `with_period(period, value)`

Override value at a specific period (zero-indexed).

Creates a modified version of a list column with a specific element set to a new value. Essential for modeling planned policy changes, premium holidays, benefit adjustments, and other known discontinuities in actuarial projections.

This method only works with list columns. For scalar columns, use conditional logic with `.when()` and `.then()`.

When to use

- **Premium Holidays:** Model scheduled breaks in premium payments, such as waiver of premium periods or contractual payment holidays.
- **Benefit Changes:** Implement known benefit adjustments at specific durations, like step-up death benefits or maturity bonuses.
- **Policy Events:** Model surrender charge schedules, conversion options, or guaranteed insurability riders that activate at specific times.
- **Assumption Overrides:** Apply one-time adjustments to mortality rates, lapse rates, or expenses for specific policy anniversaries.

##### Parameters

period : int Zero-based index to modify. Negative indices supported (-1 = last period). value : float or str Value to set at that period

##### Returns

```text
Modified list with value changed at specified period
```

##### Raises

```text
RuntimeError: If proxy not associated with an ActuarialFrame
ValueError: If period is out of bounds for the list
```

##### Examples

**Vector Example: Premium Holiday**

```python
from gaspatchio import ActuarialFrame

data = {"premium": [[1000, 1000, 1000]]}
af = ActuarialFrame(data)

af.premium_adj = af.premium.projection.with_period(1, value=0)

print(af.collect())
```

```text
shape: (1, 2)
┌────────────────────┬─────────────────┐
│ premium            ┆ premium_adj     │
│ ---                ┆ ---             │
│ list[i64]          ┆ list[i64]       │
╞════════════════════╪═════════════════╡
│ [1000, 1000, 1000] ┆ [1000, 0, 1000] │
└────────────────────┴─────────────────┘
```

**Vector Example: Negative Index (Last Period)**

```python
from gaspatchio import ActuarialFrame

data = {"benefit": [[1000, 1000, 1000]]}
af = ActuarialFrame(data)

af.benefit_adj = af.benefit.projection.with_period(-1, value=5000)

print(af.collect())
```

```text
shape: (1, 2)
┌────────────────────┬────────────────────┐
│ benefit            ┆ benefit_adj        │
│ ---                ┆ ---                │
│ list[i64]          ┆ list[i64]          │
╞════════════════════╪════════════════════╡
│ [1000, 1000, 1000] ┆ [1000, 1000, 5000] │
└────────────────────┴────────────────────┘
```

**Vector Example: Benefit Increase**

```python
from gaspatchio import ActuarialFrame

data = {"face_amount": [[100000, 100000, 100000]]}
af = ActuarialFrame(data)

af.face_adj = af.face_amount.projection.with_period(1, value=150000)

print(af.collect())
```

```text
shape: (1, 2)
┌──────────────────────────┬──────────────────────────┐
│ face_amount              ┆ face_adj                 │
│ ---                      ┆ ---                      │
│ list[i64]                ┆ list[i64]                │
╞══════════════════════════╪══════════════════════════╡
│ [100000, 100000, 100000] ┆ [100000, 150000, 100000] │
└──────────────────────────┴──────────────────────────┘
```

### `with_periods(updates)`

Override values at multiple specific periods.

Creates a modified version of a list column with multiple elements changed at once. More efficient and readable than chaining multiple `with_period()` calls. Essential for modeling complex benefit schedules, premium patterns, and assumption variations across policy durations.

When to use

- **Benefit Schedules:** Model policies with multiple benefit changes, such as increasing term insurance or scheduled death benefit steps.
- **Premium Patterns:** Implement complex premium schedules with multiple holidays, increases, or decreases at known policy anniversaries.
- **Surrender Charges:** Define surrender charge schedules that decrease over time or change at specific durations.
- **Assumption Testing:** Apply multiple one-time adjustments to test sensitivity to assumption changes at different policy durations.

##### Parameters

updates : dict[int, int | float | str] Dictionary mapping period indices (zero-based) to new values. Negative indices are supported (-1 = last period).

##### Returns

```text
Modified list with values changed at specified periods
```

##### Raises

```text
RuntimeError: If proxy not associated with an ActuarialFrame
ValueError: If any period is out of bounds for the list
```

##### Examples

**Vector Example: Multiple Premium Holidays**

```python
from gaspatchio import ActuarialFrame

data = {"premium": [[500, 500, 500]]}
af = ActuarialFrame(data)

af.premium_adj = af.premium.projection.with_periods({0: 0, 2: 0})

print(af.collect())
```

```text
shape: (1, 2)
┌─────────────────┬─────────────┐
│ premium         ┆ premium_adj │
│ ---             ┆ ---         │
│ list[i64]       ┆ list[i64]   │
╞═════════════════╪═════════════╡
│ [500, 500, 500] ┆ [0, 500, 0] │
└─────────────────┴─────────────┘
```

**Vector Example: Benefit Schedule**

```python
from gaspatchio import ActuarialFrame

data = {"benefit": [[1000, 1000, 1000]]}
af = ActuarialFrame(data)

af.benefit_adj = af.benefit.projection.with_periods({0: 1500, -1: 5000})

print(af.collect())
```

```text
shape: (1, 2)
┌────────────────────┬────────────────────┐
│ benefit            ┆ benefit_adj        │
│ ---                ┆ ---                │
│ list[i64]          ┆ list[i64]          │
╞════════════════════╪════════════════════╡
│ [1000, 1000, 1000] ┆ [1500, 1000, 5000] │
└────────────────────┴────────────────────┘
```

# Rollforward API

The rollforward runs state-machine projections across every policy in parallel. The public surface is four pieces: a builder for declaring the period-by-period steps, a compile step, a collector for extracting per-state and per-increment expressions, and the compiled model itself with its inspection helpers.

For concept-level material, see [Rollforward](https://gaspatchio.dev/0.9.0/concepts/rollforward/index.md). For runnable patterns, see `gaspatchio/tutorials/rollforward-patterns/` in the source tree.

## RollforwardBuilder

## `gaspatchio.RollforwardBuilder`

Mutable builder that produces an immutable IR on `._build()`.

## compile_rollforward

## `gaspatchio.compile_rollforward(target)`

Run the 5-pass chain over a Builder or an IR.

Each pass logs a one-line TRACE record for observability:

```text
[validate]              ok — N transitions
[resolve_state_refs]    ok
[fold_constants]        ok
[assign_capture_slots]  ok — N slots
[lower_polars]          ok — N kwargs
```

## CompiledRollforward

## `gaspatchio.CompiledRollforward`

Frozen artefact carrying the compiled IR and inspection surface.

Returned by :func:`compile_rollforward`. Carries everything the kernel needs to execute (`plugin_kwargs`, `plugin_args`, `capture_slots`), the expression surface (`expr_for`, `increment_for`), and three inspection helpers for governance and audit.

### `canonical_form()`

Return a stable, deterministic dict describing the model structure.

Two compiled rollforwards with the same Op chain (in the same order), same states, same Schedule canonical-form, and same configuration produce equal canonical-form dicts. Column-name aliases inside Op expressions are reduced to `str(expr)`, so renaming a column does not change the canonical form.

### `explain()`

Return a multi-line human-readable summary of the model.

Lists states, points, schedule, transitions in order, and the cross-cutting configuration (lapse, contract boundary, increment tracking). Plain text — fits in audit reports and TRACE logs.

### `expr_for(state, *, point='eop')`

Return a Polars Expr selecting the per-period values for (state, point).

All extractions from one compiled rollforward share a single kernel call when assigned to an `ActuarialFrame`::

```text
af.fund = compiled.expr_for("fund")
af.gmdb = compiled.expr_for("gmdb")  # no second kernel run
```

### `fingerprint()`

Return a SHA-256 fingerprint of the canonical form.

Stable across runs and across machines for an unchanged model. Suitable for governance metadata and run logs.

### `increment_for(label)`

Return a Polars Expr selecting the per-period delta for a labelled Op.

The delta is signed — what the op actually applied to its target: negative for charges, positive for credits, zero after a stop or lapse (the kernel applied nothing, where a source sheet may keep computing a notional charge against the full face amount).

### `plugin_expr()`

Return the raw kernel call as a self-contained Polars expression.

The escape hatch for use outside `ActuarialFrame`: alias the struct onto a plain LazyFrame yourself, then extract fields from that column::

```text
df = df.with_columns(compiled.plugin_expr().alias("rf"))
df = df.with_columns(av=pl.col("rf").struct.field("av@eop"))
```

Inside `ActuarialFrame` prefer :meth:`expr_for`, which shares one kernel call across every extraction automatically.

## RollforwardCollector

## `gaspatchio.RollforwardCollector`

Deprecated: use :meth:`CompiledRollforward.expr_for` instead.

Emits self-contained per-state / per-increment plugin exprs (one kernel call EACH — no sharing). Retained for backwards compatibility and for raw Polars frames, where a self-contained expr is the only thing that works.

### `expr_for(state, *, point='eop')`

Return a self-contained Expr for (state, point) — one kernel call each.

### `increment_for(label)`

Return a self-contained Expr for a labelled Op's per-period delta.

The delta is signed — what the op actually applied to its target: negative for charges, positive for credits, zero after a stop or lapse (the kernel applied nothing, where a source sheet may keep computing a notional charge).

# Scenarios API

API reference for scenario plans, the bounded-memory loop, aggregators, and shock operations.

## Scenario Plan

### ScenarioRun

### ScenarioResult

## Scenario Loop

### for_each_scenario

### BatchSnapshot

The snapshot passed to an `on_batch` callback — by `for_each_scenario` and by `ScenarioRun.run` — after every batch folds: the running aggregate partials plus progress fields (`elapsed_s`, `fraction_done`, `eta_s`, `throughput`), for rendering a convergence trace. See [Watching a run converge](https://gaspatchio.dev/0.9.0/concepts/scenarios/streaming-convergence/index.md).

### with_scenarios

## Portfolio Aggregation

Memory-safe portfolio runs: fold each batch to per-period aggregates ([Aggregating at Scale](https://gaspatchio.dev/0.9.0/concepts/scenarios/aggregating-at-scale/index.md)), or spill the full per-policy grid to disk ([Streaming and Spill](https://gaspatchio.dev/0.9.0/concepts/scenarios/streaming-and-spill/index.md)).

### run_aggregated

### AggregatedResult

### run_to_parquet

### SpillResult

### SelectionDecision

How `batch_size="auto"` resolved — the chosen batch plus the measured ladder behind it. Audit output of the auto-batch search (see [Aggregating at Scale](https://gaspatchio.dev/0.9.0/concepts/scenarios/aggregating-at-scale/#letting-the-batch-size-find-itself)).

### ProbeResult

One measured rung of the auto-batch search — a probed batch size and its measured cost.

## Aggregators

The 14 built-in aggregators implement the [Aggregator](#baseaggregator) Protocol with the `.alias()`, `.over()`, and `.of()` modifiers. See the [Aggregators concept page](https://gaspatchio.dev/0.9.0/concepts/scenarios/aggregators/index.md) for usage guidance.

### Sum

Bases: `_BaseAggregator`

### Count

Bases: `_BaseAggregator`

### Min

Bases: `_BaseAggregator`

### Max

Bases: `_BaseAggregator`

### ArgMin

Bases: `_BaseAggregator`

### ArgMax

Bases: `_BaseAggregator`

### Mean

Bases: `_BaseAggregator`

### Variance

Bases: `_BaseAggregator`

### Std

Bases: `_BaseAggregator`

### Quantile

Bases: `_BaseAggregator`

### Median

Bases: `_BaseAggregator`

### CTE

Bases: `_BaseAggregator`

### QuantileRank

Bases: `_BaseAggregator`

## Per-Period Aggregators

The `Period*` family folds a per-period list column across the portfolio at each projection period, returning one value per period (vs. the scalar aggregators above, which fold to a single number). `PeriodMedian`, `PeriodQuantile`, and `PeriodCTE` are DDSketch-backed for bounded-memory tail metrics. See the [Aggregators concept page](https://gaspatchio.dev/0.9.0/concepts/scenarios/aggregators/#per-period-results-the-period-family) for usage.

### PeriodSum

Bases: `VectorAggregator`

### PeriodMean

Bases: `VectorAggregator`

### PeriodMin

Bases: `VectorAggregator`

### PeriodMax

Bases: `VectorAggregator`

### PeriodCount

Bases: `VectorAggregator`

### PeriodStd

Bases: `VectorAggregator`

### PeriodVariance

Bases: `VectorAggregator`

### PeriodMedian

Bases: `VectorAggregator`

### PeriodQuantile

Bases: `VectorAggregator`

### PeriodCTE

Bases: `VectorAggregator`

## Custom Aggregators

See the [Custom Aggregators concept page](https://gaspatchio.dev/0.9.0/concepts/scenarios/custom-aggregators/index.md) for full worked examples.

### BaseAggregator

### VectorAggregator

The base class for per-period (`Period*`) aggregators — those carrying per-period vector state.

Bases: `_BaseAggregator`

### scenario_aggregator

### register_aggregator

## Config Parsing

### parse_shock_config

### parse_scenario_config

### parse_aggregations

## Basic Shock Classes

### MultiplicativeShock

Bases: `Shock`

A shock that multiplies values by a factor.

Used for scenarios like "increase mortality by 20%" (factor=1.2) or "decrease lapse by 10%" (factor=0.9).

Parameters:

| Name     | Type    | Description                        | Default                                 |
| -------- | ------- | ---------------------------------- | --------------------------------------- |
| `factor` | `float` | The multiplicative factor to apply | *required*                              |
| `table`  | \`str   | None\`                             | Optional table name this shock targets  |
| `column` | \`str   | None\`                             | Optional column name this shock targets |

When to use

- Stress testing with percentage changes
- Sensitivity analysis on rates
- Regulatory capital scenarios (e.g., SCR shocks)

### Examples:

**20% increase in mortality:**

````python no_output_check
from gaspatchio.scenarios.shocks import MultiplicativeShock

shock = MultiplicativeShock(factor=1.2, table="mortality")

```text
**10% decrease in lapse rates:**

```python no_output_check
shock = MultiplicativeShock(factor=0.9, table="lapse")
````

## `describe() -> str`

Return description of this shock.

### AdditiveShock

Bases: `Shock`

A shock that adds a constant delta to values.

Used for scenarios like "increase discount rate by 50bps" (delta=0.005) or "decrease expense loading by 1%" (delta=-0.01).

Parameters:

| Name     | Type    | Description                    | Default                                 |
| -------- | ------- | ------------------------------ | --------------------------------------- |
| `delta`  | `float` | The additive constant to apply | *required*                              |
| `table`  | \`str   | None\`                         | Optional table name this shock targets  |
| `column` | \`str   | None\`                         | Optional column name this shock targets |

When to use

- Interest rate shocks (parallel shifts)
- Expense loading adjustments
- Basis point changes to rates

### Examples:

**Add 50bps to discount rates:**

````python no_output_check
from gaspatchio.scenarios.shocks import AdditiveShock

shock = AdditiveShock(delta=0.005, table="discount_rates")

```text
**Subtract 1% from expense loading:**

```python no_output_check
shock = AdditiveShock(delta=-0.01, table="expenses")
````

## `describe() -> str`

Return description of this shock.

### OverrideShock

Bases: `Shock`

A shock that replaces all values with a constant.

Used for scenarios like "set lapse to zero" (value=0.0) or "assume 100% mortality" (value=1.0).

Parameters:

| Name     | Type  | Description               | Default                                 |
| -------- | ----- | ------------------------- | --------------------------------------- |
| `value`  | `Any` | The constant value to set | *required*                              |
| `table`  | \`str | None\`                    | Optional table name this shock targets  |
| `column` | \`str | None\`                    | Optional column name this shock targets |

When to use

- Extreme stress scenarios
- Disabling a decrement entirely
- Testing boundary conditions

### Examples:

**Set lapse rates to zero:**

````python no_output_check
from gaspatchio.scenarios.shocks import OverrideShock

shock = OverrideShock(value=0.0, table="lapse")

```text
**Override discount rate to flat 5%:**

```python no_output_check
shock = OverrideShock(value=0.05, table="discount_rates")
````

## `describe() -> str`

Return description of this shock.

## Value Constraint Shocks

### ClipShock

Bases: `Shock`

A shock that clips (caps/floors) values to a range.

Used for scenarios like "lapse rate cannot exceed 100%" (max=1.0) or "mortality floor of 0.1%" (min=0.001). Can also combine both.

This is essential for regulatory scenarios like Solvency II SCR lapse up, where shocked values must be capped at actuarial limits.

Parameters:

| Name        | Type    | Description | Default                                               |
| ----------- | ------- | ----------- | ----------------------------------------------------- |
| `min_value` | \`float | None\`      | Optional floor value (values below are set to this)   |
| `max_value` | \`float | None\`      | Optional ceiling value (values above are set to this) |
| `table`     | \`str   | None\`      | Optional table name this shock targets                |
| `column`    | \`str   | None\`      | Optional column name this shock targets               |

When to use

- Post-shock value constraints (e.g., lapse ≤ 100%)
- Regulatory scenarios with actuarial limits
- Preventing unrealistic shocked values
- Combining with other shocks in a pipeline

### Examples:

**Cap lapse rates at 100%:**

````python no_output_check
from gaspatchio.scenarios.shocks import ClipShock

shock = ClipShock(max_value=1.0, table="lapse")

```text
**Floor mortality at 0.1%:**

```python no_output_check
shock = ClipShock(min_value=0.001, table="mortality")
````

**Clip to a range:**

`python no_output_check shock = ClipShock(min_value=0.0, max_value=1.0, table="rates")`

## `describe() -> str`

Return description of this shock.

## Composable Shocks

### PipelineShock

Bases: `Shock`

A shock that chains multiple operations in sequence.

Used for complex scenarios like "multiply by 1.5 then cap at 100%" which requires composing multiple shock operations.

The operations are applied left-to-right: the output of each shock becomes the input to the next.

Parameters:

| Name     | Type                | Description                          | Default                                 |
| -------- | ------------------- | ------------------------------------ | --------------------------------------- |
| `shocks` | `tuple[Shock, ...]` | Sequence of shocks to apply in order | `tuple()`                               |
| `table`  | \`str               | None\`                               | Optional table name this shock targets  |
| `column` | \`str               | None\`                               | Optional column name this shock targets |

When to use

- Solvency II lapse up: multiply then cap
- Complex stress scenarios with multiple transformations
- Building reusable shock combinations

### Examples:

**Solvency II lapse up (multiply by 1.5, cap at 100%):**

```python no_output_check
from gaspatchio.scenarios.shocks import (
PipelineShock,
MultiplicativeShock,
ClipShock,
)

shock = PipelineShock(
shocks=[
MultiplicativeShock(factor=1.5),
ClipShock(max_value=1.0),
],
table="lapse",
)
```

**Lapse down with floor (multiply by 0.5, floor at original - 0.2):**

This would need a custom approach for relative floors.

## `describe() -> str`

Return description of this shock.

### MaxShock

Bases: `Shock`

A shock that takes the maximum of two shock expressions.

Used for scenarios like Solvency II lapse down: "max(lapse × 0.5, lapse - 0.2)" where the result is the larger of two transformations.

Parameters:

| Name      | Type    | Description         | Default                                 |
| --------- | ------- | ------------------- | --------------------------------------- |
| `shock_a` | `Shock` | First shock option  | *required*                              |
| `shock_b` | `Shock` | Second shock option | *required*                              |
| `table`   | \`str   | None\`              | Optional table name this shock targets  |
| `column`  | \`str   | None\`              | Optional column name this shock targets |

When to use

- Solvency II lapse down: max(×0.5, -0.2)
- Taking the less severe of two shocks
- Complex regulatory scenarios

### Examples:

**Solvency II lapse down:**

```python no_output_check
from gaspatchio.scenarios.shocks import (
MaxShock,
MultiplicativeShock,
AdditiveShock,
)

shock = MaxShock(
shock_a=MultiplicativeShock(factor=0.5),
shock_b=AdditiveShock(delta=-0.2),
table="lapse",
)

## Result: max(lapse × 0.5, lapse - 0.2)

```

## `describe() -> str`

Return description of this shock.

### MinShock

Bases: `Shock`

A shock that takes the minimum of two shock expressions.

Used for scenarios where the result should be the smaller of two transformations.

Parameters:

| Name      | Type    | Description         | Default                                 |
| --------- | ------- | ------------------- | --------------------------------------- |
| `shock_a` | `Shock` | First shock option  | *required*                              |
| `shock_b` | `Shock` | Second shock option | *required*                              |
| `table`   | \`str   | None\`              | Optional table name this shock targets  |
| `column`  | \`str   | None\`              | Optional column name this shock targets |

When to use

- Taking the more severe of two shocks
- Cap scenarios (similar to ClipShock but based on transformations)
- Complex regulatory scenarios

### Examples:

**Take the lower of two mortality assumptions:**

```python no_output_check
from gaspatchio.scenarios.shocks import (
MinShock,
MultiplicativeShock,
OverrideShock,
)

shock = MinShock(
shock_a=MultiplicativeShock(factor=1.5),
shock_b=OverrideShock(value=0.1), # Cap at 10% mortality
table="mortality",
)
```

## `describe() -> str`

Return description of this shock.

## Filter Shocks

### FilteredShock

Bases: `Shock`

A shock that applies only to rows matching a filter condition (WHERE clause).

Used for dimension-filtered shocks like "increase early-duration lapse by 25%" where only rows matching the filter are modified.

This implements GSP-65: Dimension-filtered shocks.

Parameters:

| Name     | Type              | Description                         | Default                                 |
| -------- | ----------------- | ----------------------------------- | --------------------------------------- |
| `shock`  | `Shock`           | The shock to apply to matching rows | *required*                              |
| `where`  | `FilterCondition` | Filter condition dictionary         | *required*                              |
| `table`  | \`str             | None\`                              | Optional table name this shock targets  |
| `column` | \`str             | None\`                              | Optional column name this shock targets |

When to use

- Apply shocks to specific segments (e.g., early durations)
- Age-specific mortality adjustments
- Product-specific lapse stress
- Regulatory scenarios with conditional shocks

### Examples:

**Increase early-duration lapse by 25%:**

````python no_output_check
from gaspatchio.scenarios.shocks import FilteredShock, MultiplicativeShock

shock = FilteredShock(
shock=MultiplicativeShock(factor=1.25),
where={"duration": {"lte": 3}},
table="lapse",
)

```text
**Mortality shock for elderly lives:**

```python no_output_check
shock = FilteredShock(
    shock=MultiplicativeShock(factor=1.15),
    where={"attained_age": {"gte": 65}},
    table="mortality",
)
````

**Complex filter with multiple conditions:**

`python no_output_check shock = FilteredShock( shock=AdditiveShock(delta=0.02), where={"sex": "F", "smoker_status": "S"}, table="mortality", )`

## `describe() -> str`

Return description of this shock.

### TimeConditionalShock

Bases: `Shock`

A shock that applies only at specific projection times (WHEN clause).

Used for time-conditional shocks like "40% mass lapse at t=0 only" where the shock is applied based on projection period.

This implements GSP-74: Time-conditional shocks.

Parameters:

| Name          | Type              | Description                                            | Default                                 |
| ------------- | ----------------- | ------------------------------------------------------ | --------------------------------------- |
| `shock`       | `Shock`           | The shock to apply at matching times                   | *required*                              |
| `when`        | `FilterCondition` | Time condition dictionary (uses 't' column by default) | *required*                              |
| `table`       | \`str             | None\`                                                 | Optional table name this shock targets  |
| `column`      | \`str             | None\`                                                 | Optional column name this shock targets |
| `time_column` | `str`             | Column name for time (default: "t")                    | `'t'`                                   |

When to use

- Mass lapse at policy inception (t=0)
- First-year expense shocks
- Time-limited stress scenarios
- Shock only during specific periods

### Examples:

**Mass lapse at t=0:**

````python no_output_check
from gaspatchio.scenarios.shocks import TimeConditionalShock, AdditiveShock

shock = TimeConditionalShock(
shock=AdditiveShock(delta=0.40), # Add 40% lapse
when={"t": {"eq": 0}},
table="lapse",
)

```text
**Expense shock for first 5 years:**

```python no_output_check
shock = TimeConditionalShock(
    shock=MultiplicativeShock(factor=1.10),
    when={"t": {"lte": 5}},
    table="expenses",
)
````

## `describe() -> str`

Return description of this shock.

## Parameter Shocks

### ParameterShock

A shock specification for scalar model parameters (not table values).

Used for scenarios like "increase expense inflation by 1%" where the target is a scalar model input rather than an assumption table.

Unlike table shocks, parameter shocks store the transformation specification and are applied at model setup time by the model code.

Parameters:

| Name        | Type    | Description                                          | Default    |
| ----------- | ------- | ---------------------------------------------------- | ---------- |
| `param`     | `str`   | Name of the parameter to shock                       | *required* |
| `operation` | `str`   | Type of operation ("multiply", "add", or "set")      | *required* |
| `value`     | `float` | Value for the operation (factor, delta, or constant) | *required* |

When to use

- Shocking scalar model inputs (expense inflation, discount rate spread)
- Parameters that aren't stored in assumption tables
- Model-level sensitivity analysis

Not a Shock subclass

ParameterShock is NOT a Shock subclass because it doesn't operate on Polars expressions. It stores the shock specification for the model code to apply.

### Examples:

**Add 1% to expense inflation:**

````python no_output_check
from gaspatchio.scenarios.shocks import ParameterShock

shock = ParameterShock(param="expense_inflation", operation="add", value=0.01)

## Apply in model code:

base_inflation = 0.02
shocked_inflation = shock.apply(base_inflation) # 0.03

```text
**Multiply discount spread:**

```python no_output_check
shock = ParameterShock(param="discount_spread", operation="multiply", value=1.5)
````

## `apply(base_value: float) -> float`

Apply this shock to a base parameter value.

Parameters:

| Name         | Type    | Description                  | Default    |
| ------------ | ------- | ---------------------------- | ---------- |
| `base_value` | `float` | The original parameter value | *required* |

Returns:

| Type    | Description                 |
| ------- | --------------------------- |
| `float` | The shocked parameter value |

## `describe() -> str`

Return a human-readable description of this shock.

## Related Table Methods

The following methods on the `Table` class work with scenarios:

- **`Table.from_scenario_files()`** - Load separate assumption files per scenario
- **`Table.from_scenario_template()`** - Load scenario files using a path template
- **`Table.from_shocks()`** - Create multiple shocked tables from shock specifications
- **`Table.with_shock()`** - Apply a single shock to create a modified table

See the [Actuarial Frame API](https://gaspatchio.dev/0.9.0/api/actuarial_frame/index.md) for details on these methods.

## See Also

- [Shock Operations](https://gaspatchio.dev/0.9.0/concepts/scenarios/shocks/index.md) - Conceptual guide with progressive examples
- [What-If Analysis](https://gaspatchio.dev/0.9.0/concepts/scenarios/what-if/index.md) - Natural language to config translation
- [Scenarios Overview](https://gaspatchio.dev/0.9.0/concepts/scenarios/index.md) - High-level scenario concepts

# Schedule API

A `Schedule` is your projection grid — valuation date, frequency, term, calendar, day-count — as a named, reusable object. Most models declare the projection on the frame via `af.projection.set(...)` and never build a `Schedule` directly; the frame builds one for you. Reach for the API on this page when you need to share a single grid across multiple frames, lock the grid down between valuations so quarter-over-quarter results sit on an unchanged time axis, or build per-policy grids with `Schedule.from_inception(...)`.

For the actuarial framing — when to use `from_calendar_grid` vs `from_inception`, calendar / day-count selection, anniversary recognition — see [Schedules](https://gaspatchio.dev/0.9.0/concepts/schedules/index.md).

## Schedule

## `gaspatchio.Schedule`

## Calendars

## `gaspatchio.schedule.NullCalendar`

Bases: `Calendar`

## `gaspatchio.schedule.TARGET`

Bases: `Calendar`

## `gaspatchio.schedule.UnitedKingdom`

Bases: `Calendar`

## `gaspatchio.schedule.UnitedStates`

Bases: `Calendar`

## `gaspatchio.schedule.JointCalendar`

Bases: `Calendar`

## `gaspatchio.schedule.BespokeCalendar`

Bases: `Calendar`

## `gaspatchio.schedule.calendar_from_name(name)`

## Day Counts

## `gaspatchio.schedule.OneTwelfth`

Bases: `DayCount`

## `gaspatchio.schedule.Actual360`

Bases: `DayCount`

## `gaspatchio.schedule.Actual365Fixed`

Bases: `DayCount`

## `gaspatchio.schedule.ActualActualISDA`

Bases: `DayCount`

## `gaspatchio.schedule.Thirty360`

Bases: `DayCount`

## `gaspatchio.schedule.day_count_from_name(name)`

## Business-Day Conventions

## `gaspatchio.schedule.BusinessDayConvention`

Bases: `Enum`

## `gaspatchio.column.namespaces.string_proxy.StringNamespaceProxy`

A proxy for Polars expression string (str) namespace operations.

This proxy is typically accessed via the `.str` attribute of a `ColumnProxy` or `ExpressionProxy` that refers to a string or list-of-strings column within an `ActuarialFrame`. It allows for intuitive, Polars-like string manipulations while remaining integrated with the ActuarialFrame ecosystem.

It automatically handles shimming for `List[String]` columns, applying string methods element-wise to the contents of the lists.

Examples:

**Scalar Example: Uppercasing policyholder names**

This demonstrates applying a string operation to a scalar string column. We'll convert policyholder names to uppercase.

```python
from gaspatchio.frame.base import ActuarialFrame

data_for_class_doctest = {  # Renamed to avoid conflict with other examples
    "policy_holder_name": ["John Doe", "Jane Smith", "Robert Jones"],
    "policy_type_codes": [["TERM", "WL"], ["UL"], ["TERM", "CI"]],
}
af_scalar = ActuarialFrame(data_for_class_doctest)
af_upper_names = af_scalar.select(
    af_scalar["policy_holder_name"].str.to_uppercase().alias("upper_name")
)
print(af_upper_names.collect())
```

```text
shape: (3, 1)
┌──────────────┐
│ upper_name   │
│ ---          │
│ str          │
╞══════════════╡
│ JOHN DOE     │
│ JANE SMITH   │
│ ROBERT JONES │
└──────────────┘
```

**Vector (List Shimming) Example: Lowercasing policy type codes**

This demonstrates applying a string operation to a list-of-strings column. We'll convert lists of policy type codes to lowercase.

```python
from gaspatchio.frame.base import ActuarialFrame
import polars as pl

data_for_class_doctest = {
    "policy_holder_name": ["John Doe", "Jane Smith", "Robert Jones"],
    "policy_type_codes": [["TERM", "WL"], ["UL"], ["TERM", "CI"]]
}
af_vector = ActuarialFrame(data_for_class_doctest).with_columns(
    pl.col("policy_type_codes").cast(pl.List(pl.String))
)
af_lower_codes = af_vector.select(
   af_vector["policy_type_codes"].str.to_lowercase().alias("lower_codes")
)
print(af_lower_codes.collect())
```

```text
shape: (3, 1)
┌────────────────┐
│ lower_codes    │
│ ---            │
│ list[str]      │
╞════════════════╡
│ ["term", "wl"] │
│ ["ul"]         │
│ ["term", "ci"] │
└────────────────┘
```

### `contains(pattern, literal=False, strict=False)`

Checks if strings in a column contain a specified pattern.

This method searches for a pattern within string values, returning a boolean indicating if the pattern exists in each string. It's useful for filtering, data categorization, and identifying records with specific text patterns.

When to use

- Identify policies with specific riders or endorsements from description fields
- Find claims that mention particular medical conditions or causes
- Filter customer feedback containing specific keywords for risk analysis
- Segment policyholders based on address information (e.g., rural vs urban)
- Flag policies or claims with special handling notes (e.g., "legal review")
- Screen underwriting notes for high-risk indicators

Parameters:

| Name      | Type   | Description                                                                                                                                                | Default                                                                                                                                                     |
| --------- | ------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `pattern` | \`str  | Expr\`                                                                                                                                                     | The substring or regex pattern to search for. Can be a literal string (e.g., "RiderX") or a Polars expression (e.g., pl.col("other_column_with_patterns")). |
| `literal` | `bool` | If True, pattern is treated as a literal string. If False (default), pattern is treated as a regex.                                                        | `False`                                                                                                                                                     |
| `strict`  | `bool` | If True and pattern is a Polars expression, an error is raised if pattern is not a string type. If False (default), pattern is cast to string if possible. | `False`                                                                                                                                                     |

Returns:

| Name              | Type              | Description                                                                                                                                                                     |
| ----------------- | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `ExpressionProxy` | `ExpressionProxy` | A new ExpressionProxy containing a boolean Series indicating for each input string whether the pattern was found. If the input was List[String], the output will be List[bool]. |

Examples:

**Scalar Example: Identifying policies with an Accidental Death Benefit (ADB) rider**

Imagine you have a dataset of policy descriptions and you want to flag all policies that include an "ADB" rider.

```python
from gaspatchio.frame.base import ActuarialFrame

data = {
    "policy_id": ["POL001", "POL002", "POL003", "POL004"],
    "description": [
        "Term Life Plan with ADB rider",
        "Whole Life - Standard",
        "Universal Life, includes ADB rider and Accidental Death Benefit (ADB)",
        "Term Life, no Accidental Death Benefit rider",
    ],
}
af = ActuarialFrame(data)
af_with_adb_rider = af.select(
    af["description"]
    .str.contains("ADB rider", literal=True)
    .alias("has_adb_rider")
)
print(af_with_adb_rider.collect())
```

```text
shape: (4, 1)
┌───────────────┐
│ has_adb_rider │
│ ---           │
│ bool          │
╞═══════════════╡
│ true          │
│ false         │
│ true          │
│ false         │
└───────────────┘
```

**Vector Example: Checking underwriter notes for high-risk keywords**

Suppose each policy has a list of notes from underwriters. We want to check if any note for a given policy contains keywords like "medical history" or "hazardous occupation", which might indicate higher risk.

```python
from gaspatchio.frame.base import ActuarialFrame

uw_notes_data = {
    "policy_id": ["UW001", "UW002", "UW003"],
    "underwriter_notes": [
        "Standard risk. Family history clear.",
        "Applicant works in construction. Reviewed medical history: smoker.",
        "No concerning notes. Possible hazardous occupation mentioned."
    ]
}
af_notes = ActuarialFrame(uw_notes_data)
af_results = af_notes.select(
    af_notes["underwriter_notes"].str.contains("medical history").alias("mentions_medical_history"),
    af_notes["underwriter_notes"].str.contains("(?i)hazardous occupation").alias("mentions_hazardous_occupation"),
)
print(af_results.collect())
```

```text
shape: (3, 2)
┌──────────────────────────┬───────────────────────────────┐
│ mentions_medical_history ┆ mentions_hazardous_occupation │
│ ---                      ┆ ---                           │
│ bool                     ┆ bool                          │
╞══════════════════════════╪═══════════════════════════════╡
│ false                    ┆ false                         │
│ true                     ┆ false                         │
│ false                    ┆ true                          │
└──────────────────────────┴───────────────────────────────┘
```

**Using `contains` with a list of patterns (regex and literal)**

Suppose we want to check for multiple keywords in underwriter notes using both literal and regex matching.

```python
from gaspatchio.frame.base import ActuarialFrame

uw_notes_data_multi = { # Renamed to avoid conflict
    "policy_id": ["UW001", "UW002", "UW003"],
    "underwriter_notes": [
        "Standard risk. Family history clear.",
        "Applicant works in construction. Reviewed medical history: smoker.",
        "No concerning notes. Possible hazardous occupation mentioned."
    ]
}
af_multi = ActuarialFrame(uw_notes_data_multi)
af_multi_processed = af_multi.select(
    # Literal check
    af_multi["underwriter_notes"].str.contains("medical history", literal=True).alias("mentions_medical_history_literal"),
    # Regex check (case insensitive)
    af_multi["underwriter_notes"].str.contains(r"(?i)hazardous occupation").alias("mentions_hazardous_occupation_regex"),
    # Another Regex check (case insensitive) for medical history
    af_multi["underwriter_notes"].str.contains(r"(?i)medical history").alias("mentions_medical_history_regex")
)
print(af_multi_processed.collect())
```

```text
shape: (3, 3)
┌──────────────────────────────────┬─────────────────────────────────────┬────────────────────────────────┐
│ mentions_medical_history_literal ┆ mentions_hazardous_occupation_regex ┆ mentions_medical_history_regex │
│ ---                              ┆ ---                                 ┆ ---                            │
│ bool                             ┆ bool                                ┆ bool                           │
╞══════════════════════════════════╪═════════════════════════════════════╪════════════════════════════════╡
│ false                            ┆ false                               ┆ false                          │
│ true                             ┆ false                               ┆ true                           │
│ false                            ┆ true                                ┆ false                          │
└──────────────────────────────────┴─────────────────────────────────────┴────────────────────────────────┘
```

### `ends_with(suffix)`

Check if strings end with a specific substring.

This method returns a boolean expression showing whether each string value ends with the provided suffix. For columns containing `List[String]`, the check is applied to every element within each list.

When to use

- Verify that policy identifiers end with region or product codes.
- Flag claim or log entries that end with status markers like "OK" or "PENDING".
- Validate strings against suffixes supplied in another column, such as checking payout account numbers.

Parameters:

| Name     | Type  | Description | Default                                                                                                |
| -------- | ----- | ----------- | ------------------------------------------------------------------------------------------------------ |
| `suffix` | \`str | Expr\`      | The substring to test for at the end of each string. It can be a literal value or a Polars expression. |

Returns:

| Name              | Type              | Description                                                 |
| ----------------- | ----------------- | ----------------------------------------------------------- |
| `ExpressionProxy` | `ExpressionProxy` | A boolean result indicating whether each string             |
|                   | `ExpressionProxy` | ends with suffix. For list columns, the result is a list of |
|                   | `ExpressionProxy` | booleans.                                                   |

Examples:

**Scalar example – region codes**

```python
from gaspatchio.frame.base import ActuarialFrame

af = ActuarialFrame(
    {"policy_id": ["P100-US", "P101-CA", "P102-US", None, "P103-EU"]}
)
result = af.select(
    af["policy_id"].str.ends_with("-US").alias("is_us_policy")
)
print(result.collect())
```

```text
shape: (5, 1)
┌──────────────┐
│ is_us_policy │
│ ---          │
│ bool         │
╞══════════════╡
│ true         │
│ false        │
│ true         │
│ null         │
│ false        │
└──────────────┘
```

**Vector (list) example – status flags**

```python
from gaspatchio.frame.base import ActuarialFrame
import polars as pl

logs = {
    "policy_id": ["A100", "A101"],
    "update_notes_str": [
        "Issued OK,Review PENDING",
        "None,Paid OK",
    ],
}
af_logs = ActuarialFrame(logs)
af_logs = af_logs.with_columns(
    af_logs["update_notes_str"].str.split(",").alias("update_notes").map_elements(
        lambda x: [None if item == "None" else item for item in x], return_dtype=pl.List(pl.String)
    )
)
status_ok = af_logs.select(
    af_logs["update_notes"].str.ends_with("OK").alias("ends_with_ok")
)
print(status_ok.collect())
```

```text
shape: (2, 1)
┌───────────────┐
│ ends_with_ok  │
│ ---           │
│ list[bool]    │
╞═══════════════╡
│ [true, false] │
│ [null, true]  │
└───────────────┘
```

### `extract(pattern, group_index=1)`

Extract a capturing group from a regex pattern.

Return group `group_index` from each string that matches `pattern`. Works element-wise on `List[String]` columns.

When to use

- **Identifiers**: Pull policy/claim numbers from mixed strings.
- **Amounts**: Capture monetary values from notes for validation.

Parameters:

| Name          | Type  | Description                             | Default    |
| ------------- | ----- | --------------------------------------- | ---------- |
| `pattern`     | `str` | Regex with the desired capturing group. | *required* |
| `group_index` | `int` | 1-based index of the group to extract.  | `1`        |

Returns:

| Type              | Description     |
| ----------------- | --------------- |
| `ExpressionProxy` | ExpressionProxy |

Examples: **Scalar example: extract policy number**

```python
from gaspatchio import ActuarialFrame

data = {"policy_id": ["P1", "P2"], "raw": ["POL-12345-AB", "CLAIM-678-X"]}
af = ActuarialFrame(data)

af.policy_num = af.raw.str.extract(r"POL-([A-Z0-9]+)-", group_index=1)

print(af.select(af.policy_num).collect())
```

```text
shape: (2, 1)
┌────────────┐
│ policy_num │
│ ---        │
│ str        │
╞════════════╡
│ 12345      │
│ null       │
└────────────┘
```

**Vector example: extract amounts from list of notes**

```python
from gaspatchio import ActuarialFrame

data = {
    "policy_id": ["P1"],
    "txn": [["Premium $100.50", "Fee $10.00", "Adj $-5.25"]],
}
af = ActuarialFrame(data)

af.amounts = af.txn.str.extract(r"\$?([-+]?[0-9]+\.[0-9]{2})", group_index=1)

print(af.select(af.amounts).collect())
```

```text
shape: (1, 1)
┌──────────────────────────────┐
│ amounts                      │
│ ---                          │
│ list[str]                    │
╞══════════════════════════════╡
│ ["100.50", "10.00", "-5.25"] │
└──────────────────────────────┘
```

### `extract_all(pattern)`

Extract all non-overlapping regex matches as a list.

Works element-wise on `List[String]` columns.

Parameters:

| Name      | Type  | Description                  | Default    |
| --------- | ----- | ---------------------------- | ---------- |
| `pattern` | `str` | Regex pattern to search for. | *required* |

Returns:

| Type              | Description     |
| ----------------- | --------------- |
| `ExpressionProxy` | ExpressionProxy |

When to use

- **Amounts**: Collect every currency amount from notes.
- **IDs**: Gather all reference numbers embedded in text.

Examples: **Scalar example: amounts in claim descriptions**

```python
from gaspatchio import ActuarialFrame

data = {
    "claim_id": ["C1", "C2"],
    "details": ["Paid $150.00 and $25.50 fee", "Refunded $10.00"],
}
af = ActuarialFrame(data)

af.amounts = af.details.str.extract_all(r"\$[0-9]+\.[0-9]{2}")

print(af.select(af.amounts).collect())
```

```text
shape: (2, 1)
┌───────────────────────┐
│ amounts               │
│ ---                   │
│ list[str]             │
╞═══════════════════════╡
│ ["$150.00", "$25.50"] │
│ ["$10.00"]            │
└───────────────────────┘
```

**Vector example: policy numbers from list of notes**

```python
from gaspatchio import ActuarialFrame

data = {
    "policy_id": ["P1"],
    "notes": [["Policy 12345 reported", "Adjustment for policy 98765"]],
}
af = ActuarialFrame(data)

af.policy_numbers = af.notes.str.extract_all(r"[0-9]+")

print(af.select(af.policy_numbers).collect())
```

```text
shape: (1, 1)
┌────────────────────────┐
│ policy_numbers         │
│ ---                    │
│ list[list[str]]        │
╞════════════════════════╡
│ [["12345"], ["98765"]] │
└────────────────────────┘
```

### `len_bytes()`

Get the number of bytes in each string.

Calculates the byte length of each string in a column. This is particularly useful when dealing with multi-byte character encodings (like UTF-8) where the number of characters may not equal the number of bytes.

When to use

- **Data Storage Estimation:** Accurately estimating storage requirements for datasets containing text fields, especially with international character sets (e.g., policyholder names, addresses from various regions).
- **System Integration Limits:** Ensuring that string data, when exported or sent to other systems, conforms to byte-length restrictions imposed by those systems (e.g., fixed-width file formats or database field constraints defined in bytes).
- **Performance Considerations:** Recognizing that operations on strings with many multi-byte characters might be more resource-intensive.
- **Encoding Issue Detection:** While not a direct detection method, unexpected byte lengths compared to character lengths might hint at encoding problems or the presence of unusual characters.

Returns:

| Name              | Type              | Description                                                                                                                         |
| ----------------- | ----------------- | ----------------------------------------------------------------------------------------------------------------------------------- |
| `ExpressionProxy` | `ExpressionProxy` | An ExpressionProxy with the byte count (as UInt32) for each string. If the input was List[String], the output will be List[UInt32]. |

Examples:

**Scalar Example: Byte length of UTF-8 encoded client names**

Scenario: You have client names that may include characters from various languages, and you need to understand their storage size in bytes.

```python
from gaspatchio.frame.base import ActuarialFrame

data = {
    "client_id": ["C001", "C002", "C003", "C004"],
    "client_name": [
        "René",
        "沐宸",
        "Zoë",
        "John Doe",
    ],  # French, Chinese, German, English names
}
af = ActuarialFrame(data)
af_byte_len = af.select(
    af["client_name"].str.len_bytes().alias("name_byte_length")
)
print(af_byte_len.collect())
```

```text
shape: (4, 1)
┌──────────────────┐
│ name_byte_length │
│ ---              │
│ u32              │
╞══════════════════╡
│ 5                │
│ 6                │
│ 4                │
│ 8                │
└──────────────────┘
```

**Vector Example: Byte length of free-text comments in a list**

Scenario: A policy record contains a list of comments, potentially with special characters or different languages. You need to find the byte length of each comment.

```python
from gaspatchio import ActuarialFrame

data = {
    "policy_id": ["P501", "P502"],
    "comments": [
        ["Test € symbol", "Standard comment.", None],
        ["Résumé", "日本語のコメント"]
    ]
}
af = ActuarialFrame(data)

af.byte_lengths = af.comments.str.len_bytes()

print(af.collect())
```

```text
shape: (2, 3)
┌───────────┬──────────────────────────────────────────────┬────────────────┐
│ policy_id ┆ comments                                     ┆ byte_lengths   │
│ ---       ┆ ---                                          ┆ ---            │
│ str       ┆ list[str]                                    ┆ list[u32]      │
╞═══════════╪══════════════════════════════════════════════╪════════════════╡
│ P501      ┆ ["Test € symbol", "Standard comment.", null] ┆ [15, 17, null] │
│ P502      ┆ ["Résumé", "日本語のコメント"]               ┆ [8, 24]        │
└───────────┴──────────────────────────────────────────────┴────────────────┘
```

### `len_chars()`

Alias for `n_chars`. Get the number of characters in each string.

Calculates the length of each string in a column, returning an integer representing the number of characters. This is an alias for `n_chars()`.

When to use

- **Data Validation:** Ensuring identifiers like policy numbers, social security numbers, or postal codes adhere to expected length constraints, helping to identify data entry errors.
- **System Integration:** Verifying that string data, such as client names or addresses, does not exceed length limitations of downstream systems or databases.
- **Feature Engineering:** Using the length of free-text fields (e.g., claim descriptions, underwriter notes) as a potential feature in predictive models, where length might correlate with complexity or severity.
- **Data Quality Assessment:** Identifying outliers or anomalies in string lengths that might indicate corrupted or incomplete data.

Returns:

| Name              | Type              | Description                                                                                                                              |
| ----------------- | ----------------- | ---------------------------------------------------------------------------------------------------------------------------------------- |
| `ExpressionProxy` | `ExpressionProxy` | An ExpressionProxy with the character count (as UInt32) for each string. If the input was List[String], the output will be List[UInt32]. |

Examples:

**Scalar Example: Validating policy number length**

Scenario: You need to check if policy numbers in your dataset conform to an expected length, say 7 characters.

```python
from gaspatchio.frame.base import ActuarialFrame

data = {
    "policy_id_raw": ["POL1234", "POL567", "POL89012", None, "POL3456"],
    "premium": [100.0, 150.0, 200.0, 50.0, 120.0],
}
af = ActuarialFrame(data)

# Calculate the length of each policy_id_raw
af_len_check = af.select(
    af["policy_id_raw"].str.len_chars().alias("policy_id_length")
)
print(af_len_check.collect())
```

```text
shape: (5, 1)
┌──────────────────┐
│ policy_id_length │
│ ---              │
│ u32              │
╞══════════════════╡
│ 7                │
│ 6                │
│ 8                │
│ null             │
│ 7                │
└──────────────────┘
```

**Vector Example: Character count of claim notes**

Scenario: Each policy may have a list of associated claim notes. You want to find the character length of each note to understand the verbosity or for display purposes.

```python
from gaspatchio import ActuarialFrame

data = {
    "policy_id": ["P7001", "P7002"],
    "claim_notes": [
        ["Short note.", "This is a much longer note regarding the claim details.", None],
        ["Urgent review needed!", "All clear."]
    ]
}
af = ActuarialFrame(data)

af.note_lengths = af.claim_notes.str.len_chars()

print(af.collect())
```

```text
shape: (2, 3)
┌───────────┬──────────────────────────────────────────────────────────────────────────────────┬────────────────┐
│ policy_id ┆ claim_notes                                                                      ┆ note_lengths   │
│ ---       ┆ ---                                                                              ┆ ---            │
│ str       ┆ list[str]                                                                        ┆ list[u32]      │
╞═══════════╪══════════════════════════════════════════════════════════════════════════════════╪════════════════╡
│ P7001     ┆ ["Short note.", "This is a much longer note regarding the claim details.", null] ┆ [11, 55, null] │
│ P7002     ┆ ["Urgent review needed!", "All clear."]                                          ┆ [21, 10]       │
└───────────┴──────────────────────────────────────────────────────────────────────────────────┴────────────────┘
```

### `ljust(width, fill_char=' ')`

Left-align strings by padding on the right.

Strings shorter than `width` are padded on the right with `fill_char`. When the column contains `List[String]` values, each element is padded individually.

When to use

- Formatting account or policy identifiers for fixed-width exports.
- Preparing ledger extracts where text fields must be left-aligned.
- Normalizing rider or sub-account codes stored as lists so they compare consistently.

Parameters:

| Name        | Type  | Description                                           | Default    |
| ----------- | ----- | ----------------------------------------------------- | ---------- |
| `width`     | `int` | The desired total length of the string after padding. | *required* |
| `fill_char` | `str` | The character to pad with. Defaults to a space.       | `' '`      |

Returns:

| Name              | Type              | Description                                        |
| ----------------- | ----------------- | -------------------------------------------------- |
| `ExpressionProxy` | `ExpressionProxy` | An ExpressionProxy with strings padded at the end. |

Examples:

**Scalar Example: Account Codes**

```python
from gaspatchio import ActuarialFrame

data = {
    "policy_id": ["P001", "P002", "P003", "P004"],
    "account_code": ["A1", "B123", None, "C"],
}
af = ActuarialFrame(data)

af.ljust_code = af.account_code.str.ljust(6, "-")

print(af.collect())
```

```text
shape: (4, 3)
┌───────────┬──────────────┬────────────┐
│ policy_id ┆ account_code ┆ ljust_code │
│ ---       ┆ ---          ┆ ---        │
│ str       ┆ str          ┆ str        │
╞═══════════╪══════════════╪════════════╡
│ P001      ┆ A1           ┆ A1----     │
│ P002      ┆ B123         ┆ B123--     │
│ P003      ┆ null         ┆ null       │
│ P004      ┆ C            ┆ C-----     │
└───────────┴──────────────┴────────────┘
```

**Vector Example: Sub Codes**

```python
from gaspatchio import ActuarialFrame

data = {
    "policy_id": ["P001"],
    "sub_codes": [["S1", "LONGCODE", "S23"]],
}
af = ActuarialFrame(data)

af.ljust_sub_codes = af.sub_codes.str.ljust(8, "X")

print(af.collect())
```

```text
shape: (1, 3)
┌───────────┬───────────────────────────┬──────────────────────────────────────┐
│ policy_id ┆ sub_codes                 ┆ ljust_sub_codes                      │
│ ---       ┆ ---                       ┆ ---                                  │
│ str       ┆ list[str]                 ┆ list[str]                            │
╞═══════════╪═══════════════════════════╪══════════════════════════════════════╡
│ P001      ┆ ["S1", "LONGCODE", "S23"] ┆ ["S1XXXXXX", "LONGCODE", "S23XXXXX"] │
└───────────┴───────────────────────────┴──────────────────────────────────────┘
```

### `n_chars()`

Get the number of characters in each string.

This function calculates the length of each string in a column, returning an integer representing the number of characters. It's a fundamental operation for understanding string data characteristics.

When to use

- **Data Quality Checks:** Identifying unexpectedly short or long strings that might indicate data entry errors or truncation (e.g., validating the length of policy numbers, postal codes, or identification numbers).
- **Feature Engineering:** Creating new features based on string length for predictive models (e.g., the length of a claim description might correlate with claim complexity).
- **Data Cleaning & Transformation:** Deciding on padding or truncation strategies if string fields need to conform to a fixed length for system integration or reporting.
- **Understanding Free-Text Fields:** Analyzing the distribution of lengths in fields like underwriter notes or medical descriptions to gauge the amount of detail typically provided.
- **Filtering or Segmenting Data:** Selecting records based on the length of a specific string field (e.g., finding all policyholder names shorter than 3 characters for review).

Returns:

| Name              | Type              | Description                                                              |
| ----------------- | ----------------- | ------------------------------------------------------------------------ |
| `ExpressionProxy` | `ExpressionProxy` | An ExpressionProxy with the character count (as UInt32) for each string. |

Examples:

**Scalar Example: Length of product names**

To understand the typical length of product names in your portfolio, or to identify names that might be too long for certain display formats.

```python
from gaspatchio.frame.base import ActuarialFrame

data = {
    "product_code": ["L-TERM-10", "L-WL-P", "ANN-SDA"],
    "product_name": [
        "Term Life 10 Year",
        "Whole Life Par",
        "Single Deferred Annuity",
    ],
}
af = ActuarialFrame(data)
af_len = af.select(af["product_name"].str.n_chars().alias("name_length"))
print(af_len.collect())
```

```text
shape: (3, 1)
┌─────────────┐
│ name_length │
│ ---         │
│ u32         │
╞═════════════╡
│ 17          │
│ 14          │
│ 23          │
└─────────────┘
```

**Vector Example: Length of beneficiary names in a list**

For policies with multiple beneficiaries, you might want to check the length of each beneficiary's name, perhaps to ensure it fits within system limits or for data validation.

```python
from gaspatchio import ActuarialFrame

data = {
    "policy_id": ["P001", "P002"],
    "beneficiaries": [["John A. Doe", "Jane B. Smith"], ["Robert King", None, "Alice Wonderland"]]
}
af = ActuarialFrame(data)

af.name_lengths = af.beneficiaries.str.n_chars()

print(af.collect())
```

```text
shape: (2, 3)
┌───────────┬───────────────────────────────────────────┬────────────────┐
│ policy_id ┆ beneficiaries                             ┆ name_lengths   │
│ ---       ┆ ---                                       ┆ ---            │
│ str       ┆ list[str]                                 ┆ list[u32]      │
╞═══════════╪═══════════════════════════════════════════╪════════════════╡
│ P001      ┆ ["John A. Doe", "Jane B. Smith"]          ┆ [11, 13]       │
│ P002      ┆ ["Robert King", null, "Alice Wonderland"] ┆ [11, null, 16] │
└───────────┴───────────────────────────────────────────┴────────────────┘
```

### `pad_end(width, fill_char=' ')`

Left-align strings by padding on the right.

Strings shorter than `width` are padded on the right with `fill_char`. If the column is `List[String]` the padding is applied to each element of the list.

When to use

- Format policy numbers or claim identifiers for extracts that require fixed-width fields.
- Pad abbreviations in list columns (such as rider codes) so that they line up cleanly in cross-system feeds.

Parameters:

| Name        | Type  | Description                                           | Default    |
| ----------- | ----- | ----------------------------------------------------- | ---------- |
| `width`     | `int` | The desired total length of the string after padding. | *required* |
| `fill_char` | `str` | The character to pad with. Defaults to a space.       | `' '`      |

Returns:

| Name              | Type              | Description                                        |
| ----------------- | ----------------- | -------------------------------------------------- |
| `ExpressionProxy` | `ExpressionProxy` | An ExpressionProxy with strings padded at the end. |

Examples:

**Scalar Example: Policy Codes**

```python
from gaspatchio import ActuarialFrame

data = {
    "policy_id": ["P001", "P002", "P003"],
    "policy_code": ["L101", "L20", None],
}
af = ActuarialFrame(data)

af.fixed_length_code = af.policy_code.str.pad_end(6, "0")

print(af.collect())
```

```text
shape: (3, 3)
┌───────────┬─────────────┬───────────────────┐
│ policy_id ┆ policy_code ┆ fixed_length_code │
│ ---       ┆ ---         ┆ ---               │
│ str       ┆ str         ┆ str               │
╞═══════════╪═════════════╪═══════════════════╡
│ P001      ┆ L101        ┆ L10100            │
│ P002      ┆ L20         ┆ L20000            │
│ P003      ┆ null        ┆ null              │
└───────────┴─────────────┴───────────────────┘
```

**Vector Example: Claim Codes**

```python
from gaspatchio import ActuarialFrame

data = {
    "policy_id": ["P001"],
    "claim_codes": [["A1", "XYZ", "C1234"]],
}
af = ActuarialFrame(data)

af.aligned_codes = af.claim_codes.str.pad_end(6, "_")

print(af.collect())
```

```text
shape: (1, 3)
┌───────────┬────────────────────────┬────────────────────────────────┐
│ policy_id ┆ claim_codes            ┆ aligned_codes                  │
│ ---       ┆ ---                    ┆ ---                            │
│ str       ┆ list[str]              ┆ list[str]                      │
╞═══════════╪════════════════════════╪════════════════════════════════╡
│ P001      ┆ ["A1", "XYZ", "C1234"] ┆ ["A1____", "XYZ___", "C1234_"] │
└───────────┴────────────────────────┴────────────────────────────────┘
```

### `pad_start(width, fill_char=' ')`

Alias for `rjust`. Pads the start of strings (right-aligns content).

Adds characters to the beginning of each string until it reaches the given width. This is handy when preparing fixed-width extracts or aligning numeric text fields in actuarial reports.

When to use

- Preparing policy identifiers for legacy mainframe interfaces that expect fixed-width fields.
- Aligning premium or reserve amounts in textual summaries generated for regulators or management.
- Standardizing rider codes stored in lists so that they can be compared consistently across policies.

Parameters:

| Name        | Type  | Description                                     | Default    |
| ----------- | ----- | ----------------------------------------------- | ---------- |
| `width`     | `int` | The desired minimum length of the string.       | *required* |
| `fill_char` | `str` | The character to pad with. Defaults to a space. | `' '`      |

Returns:

| Name              | Type              | Description                                   |
| ----------------- | ----------------- | --------------------------------------------- |
| `ExpressionProxy` | `ExpressionProxy` | An ExpressionProxy with strings padded at the |
|                   | `ExpressionProxy` | start.                                        |

Examples:

**Scalar Example: Premium Amounts**

```python
from gaspatchio import ActuarialFrame

data = {
    "policy_id": ["P001", "P002", "P003"],
    "premium_str": ["1200.5", "85.75", None],
}
af = ActuarialFrame(data)

af.padded_premium = af.premium_str.str.pad_start(8, " ")

print(af.collect())
```

```text
shape: (3, 3)
┌───────────┬─────────────┬────────────────┐
│ policy_id ┆ premium_str ┆ padded_premium │
│ ---       ┆ ---         ┆ ---            │
│ str       ┆ str         ┆ str            │
╞═══════════╪═════════════╪════════════════╡
│ P001      ┆ 1200.5      ┆   1200.5       │
│ P002      ┆ 85.75       ┆    85.75       │
│ P003      ┆ null        ┆ null           │
└───────────┴─────────────┴────────────────┘
```

**Vector Example: Rider Codes**

```python
from gaspatchio import ActuarialFrame

data = {
    "policy_id": ["P001"],
    "rider_codes": [["RID1", "LONGRID", "R2"]],
}
af = ActuarialFrame(data)

af.padded_codes = af.rider_codes.str.pad_start(8, "0")

print(af.collect())
```

```text
shape: (1, 3)
┌───────────┬───────────────────────────┬──────────────────────────────────────┐
│ policy_id ┆ rider_codes               ┆ padded_codes                         │
│ ---       ┆ ---                       ┆ ---                                  │
│ str       ┆ list[str]                 ┆ list[str]                            │
╞═══════════╪═══════════════════════════╪══════════════════════════════════════╡
│ P001      ┆ ["RID1", "LONGRID", "R2"] ┆ ["0000RID1", "0LONGRID", "000000R2"] │
└───────────┴───────────────────────────┴──────────────────────────────────────┘
```

### `remove_prefix(prefix)`

Alias for `strip_prefix`. Remove a prefix from each string.

The prefix is removed from the beginning of every string. Strings without that prefix remain unchanged. `List[String]` columns are processed element by element.

When to use

- **Standardizing vendor codes** before mapping them to your base product dictionary.
- **Cleaning temporary policy identifiers** created during data migrations.
- **Dropping country prefixes** from location codes when you need only the state or province.

Parameters:

| Name     | Type  | Description | Default                                                                             |
| -------- | ----- | ----------- | ----------------------------------------------------------------------------------- |
| `prefix` | \`str | Expr\`      | The substring to remove. May be a literal string or an expression resolving to one. |

Returns:

| Name              | Type              | Description                             |
| ----------------- | ----------------- | --------------------------------------- |
| `ExpressionProxy` | `ExpressionProxy` | The expression with the prefix removed. |

Examples:

**Scalar Example: Policy IDs**

```python
from gaspatchio import ActuarialFrame

data = {
    "policy_id": ["P001", "P002", "P003", "P004"],
    "policy_id_raw": ["TMP-001", "TMP-002", "003", None],
}
af = ActuarialFrame(data)

af.policy_id_clean = af.policy_id_raw.str.remove_prefix("TMP-")

print(af.collect())
```

```text
shape: (4, 3)
┌───────────┬───────────────┬─────────────────┐
│ policy_id ┆ policy_id_raw ┆ policy_id_clean │
│ ---       ┆ ---           ┆ ---             │
│ str       ┆ str           ┆ str             │
╞═══════════╪═══════════════╪═════════════════╡
│ P001      ┆ TMP-001       ┆ 001             │
│ P002      ┆ TMP-002       ┆ 002             │
│ P003      ┆ 003           ┆ 003             │
│ P004      ┆ null          ┆ null            │
└───────────┴───────────────┴─────────────────┘
```

**Vector Example: Feature Codes**

```python
from gaspatchio import ActuarialFrame

data = {
    "policy_id": ["P001", "P002"],
    "feature_codes": [
        ["LEGACY-RIDER1", "BENEFIT_A"],
        ["LEGACY-OPTION_B"],
    ],
}
af = ActuarialFrame(data)

af.clean_codes = af.feature_codes.str.remove_prefix("LEGACY-")

print(af.collect())
```

```text
shape: (2, 3)
┌───────────┬────────────────────────────────┬─────────────────────────┐
│ policy_id ┆ feature_codes                  ┆ clean_codes             │
│ ---       ┆ ---                            ┆ ---                     │
│ str       ┆ list[str]                      ┆ list[str]               │
╞═══════════╪════════════════════════════════╪═════════════════════════╡
│ P001      ┆ ["LEGACY-RIDER1", "BENEFIT_A"] ┆ ["RIDER1", "BENEFIT_A"] │
│ P002      ┆ ["LEGACY-OPTION_B"]            ┆ ["OPTION_B"]            │
└───────────┴────────────────────────────────┴─────────────────────────┘
```

### `remove_suffix(suffix)`

Alias for `strip_suffix`. Remove a suffix from each string.

This method behaves identically to meth:`strip_suffix`, removing the specified trailing substring from each string value. If a string does not end with the provided suffix it is returned unchanged. When the column is a list of strings, the removal is applied element-wise.

When to use

- **Normalizing Product Names:** Stripping version tags like "-2024" or "\_NEW" from product identifiers so that experience can be grouped by the base product.
- **Cleaning Import Data:** Eliminating temporary indicators such as "-DRAFT" that may be appended to policy numbers imported from administration systems.
- **Simplifying Text Fields:** Removing trailing notes like "\*cancelled" from agent remarks prior to text analytics or matching.

Parameters:

| Name     | Type  | Description | Default                                                                                          |
| -------- | ----- | ----------- | ------------------------------------------------------------------------------------------------ |
| `suffix` | \`str | Expr\`      | The suffix to remove. Can be a literal string or a Polars expression that evaluates to a string. |

Returns:

| Name              | Type              | Description                                    |
| ----------------- | ----------------- | ---------------------------------------------- |
| `ExpressionProxy` | `ExpressionProxy` | A new ExpressionProxy with the suffix removed. |

Examples:

**Scalar Example: Policy Codes**

```python
from gaspatchio import ActuarialFrame

data = {
    "policy_id": ["P001", "P002", "P003"],
    "policy_code": ["TERM10-OLD", "WL-OLD", "ANN"],
}
af = ActuarialFrame(data)

af.code_clean = af.policy_code.str.remove_suffix("-OLD")

print(af.collect())
```

```text
shape: (3, 3)
┌───────────┬─────────────┬────────────┐
│ policy_id ┆ policy_code ┆ code_clean │
│ ---       ┆ ---         ┆ ---        │
│ str       ┆ str         ┆ str        │
╞═══════════╪═════════════╪════════════╡
│ P001      ┆ TERM10-OLD  ┆ TERM10     │
│ P002      ┆ WL-OLD      ┆ WL         │
│ P003      ┆ ANN         ┆ ANN        │
└───────────┴─────────────┴────────────┘
```

**Vector Example: Underwriting Notes**

```python
from gaspatchio import ActuarialFrame

data = {
    "policy_id": ["P001", "P002"],
    "uw_notes": [
        ["Declined*exp", "Check later*exp"],
        ["Approved"],
    ],
}
af = ActuarialFrame(data)

af.notes_clean = af.uw_notes.str.remove_suffix("*exp")

print(af.collect())
```

```text
shape: (2, 3)
┌───────────┬─────────────────────────────────────┬─────────────────────────────┐
│ policy_id ┆ uw_notes                            ┆ notes_clean                 │
│ ---       ┆ ---                                 ┆ ---                         │
│ str       ┆ list[str]                           ┆ list[str]                   │
╞═══════════╪═════════════════════════════════════╪═════════════════════════════╡
│ P001      ┆ ["Declined*exp", "Check later*exp"] ┆ ["Declined", "Check later"] │
│ P002      ┆ ["Approved"]                        ┆ ["Approved"]                │
└───────────┴─────────────────────────────────────┴─────────────────────────────┘
```

### `replace(pattern, value, literal=False, n=1)`

Replace occurrences of a pattern in each string.

Search each string for a substring or regex and replace up to `n` matches with `value`. If `literal` is `True` the `pattern` is treated as plain text; otherwise it is interpreted as a regex.

When to use

- **Normalize legacy codes**: Map outdated product/policy codes to your current standard before joining to assumptions.
- **Clean underwriting/claim notes**: Remove boilerplate prefixes or redact sensitive fragments prior to text analysis.
- **Harmonize reference data**: Align naming conventions across multiple admin systems before aggregation.

Parameters:

| Name      | Type   | Description                                            | Default                                                                                         |
| --------- | ------ | ------------------------------------------------------ | ----------------------------------------------------------------------------------------------- |
| `pattern` | \`str  | Expr\`                                                 | Substring or regex pattern to search for. May also be a Polars expression yielding the pattern. |
| `value`   | \`str  | Expr\`                                                 | Replacement text. Can be a string or a Polars expression.                                       |
| `literal` | `bool` | If True, treat pattern as a literal string.            | `False`                                                                                         |
| `n`       | `int`  | Maximum number of replacements per string (default 1). | `1`                                                                                             |

Returns:

| Type              | Description                                                         |
| ----------------- | ------------------------------------------------------------------- |
| `ExpressionProxy` | ExpressionProxy Expression with the specified replacements applied. |

##### Examples:

**Scalar example: normalize policy status**

```python
from gaspatchio import ActuarialFrame

data = {
    "policy_id": ["P1", "P2", "P3"],
    "status_raw": ["IN FORCE", "LAPSED", "IN FORCE"],
}
af = ActuarialFrame(data)

af.status = af.status_raw.str.replace("IN FORCE", "INFORCE", literal=True)

print(af.collect())
```

```text
shape: (3, 3)
┌───────────┬────────────┬─────────┐
│ policy_id ┆ status_raw ┆ status  │
│ ---       ┆ ---        ┆ ---     │
│ str       ┆ str        ┆ str     │
╞═══════════╪════════════╪═════════╡
│ P1        ┆ IN FORCE   ┆ INFORCE │
│ P2        ┆ LAPSED     ┆ LAPSED  │
│ P3        ┆ IN FORCE   ┆ INFORCE │
└───────────┴────────────┴─────────┘
```

**Vector example: remove "NOTE: " from claim notes**

```python
from gaspatchio import ActuarialFrame

data = {
    "policy_id": ["A1", "A2"],
    "claim_notes": [["NOTE: Initial review", "Payment authorised"], [None, "NOTE: Follow up required"]],
}
af = ActuarialFrame(data)

af.clean_notes = af.claim_notes.str.replace("NOTE: ", "", literal=True, n=1)

print(af.collect())
```

```text
shape: (2, 3)
┌───────────┬────────────────────────────────────────────────┬──────────────────────────────────────────┐
│ policy_id ┆ claim_notes                                    ┆ clean_notes                              │
│ ---       ┆ ---                                            ┆ ---                                      │
│ str       ┆ list[str]                                      ┆ list[str]                                │
╞═══════════╪════════════════════════════════════════════════╪══════════════════════════════════════════╡
│ A1        ┆ ["NOTE: Initial review", "Payment authorised"] ┆ ["Initial review", "Payment authorised"] │
│ A2        ┆ [null, "NOTE: Follow up required"]             ┆ [null, "Follow up required"]             │
└───────────┴────────────────────────────────────────────────┴──────────────────────────────────────────┘
```

### `rjust(width, fill_char=' ')`

Right-align strings by padding on the left.

Strings shorter than `width` are padded on the left with `fill_char`. If the column is `List[String]` the padding is applied to each element of the list.

When to use

- Aligning premium or claim amounts before exporting to legacy ledger systems.
- Presenting policy identifiers or rider codes in uniformly padded columns for regulatory or management reports.

Parameters:

| Name        | Type  | Description                                           | Default    |
| ----------- | ----- | ----------------------------------------------------- | ---------- |
| `width`     | `int` | The desired total length of the string after padding. | *required* |
| `fill_char` | `str` | The character to pad with. Defaults to a space.       | `' '`      |

Returns:

| Name              | Type              | Description                                          |
| ----------------- | ----------------- | ---------------------------------------------------- |
| `ExpressionProxy` | `ExpressionProxy` | An ExpressionProxy with strings padded at the start. |

Examples:

**Scalar Example: Premium Amounts**

```python
from gaspatchio import ActuarialFrame

data = {
    "policy_id": ["P001", "P002", "P003"],
    "premium_str": ["123.45", "7", None],
}
af = ActuarialFrame(data)

af.rjust_premium = af.premium_str.str.rjust(8)

print(af.collect())
```

```text
shape: (3, 3)
┌───────────┬─────────────┬───────────────┐
│ policy_id ┆ premium_str ┆ rjust_premium │
│ ---       ┆ ---         ┆ ---           │
│ str       ┆ str         ┆ str           │
╞═══════════╪═════════════╪═══════════════╡
│ P001      ┆ 123.45      ┆   123.45      │
│ P002      ┆ 7           ┆        7      │
│ P003      ┆ null        ┆ null          │
└───────────┴─────────────┴───────────────┘
```

**Vector Example: Claim References**

```python
from gaspatchio import ActuarialFrame

data = {
    "policy_id": ["P001"],
    "claim_refs": [["C1", "C234", "C56789"]],
}
af = ActuarialFrame(data)

af.formatted_refs = af.claim_refs.str.rjust(6, "0")

print(af.collect())
```

```text
shape: (1, 3)
┌───────────┬──────────────────────────┬────────────────────────────────┐
│ policy_id ┆ claim_refs               ┆ formatted_refs                 │
│ ---       ┆ ---                      ┆ ---                            │
│ str       ┆ list[str]                ┆ list[str]                      │
╞═══════════╪══════════════════════════╪════════════════════════════════╡
│ P001      ┆ ["C1", "C234", "C56789"] ┆ ["0000C1", "00C234", "C56789"] │
└───────────┴──────────────────────────┴────────────────────────────────┘
```

### `starts_with(prefix)`

Check if strings in a column start with a given substring.

This is useful for categorizing or flagging records based on prefixes in textual data. For example, identifying policies based on product code prefixes (e.g., "TERM-" for term life, "WL-" for whole life) or segmenting claims by a prefix in their claim ID (e.g., "AUTO-" for auto claims).

When applied to a column of `List[String]`, such as a list of associated product features for a policy, the operation is performed element-wise on each string within each list, returning a list of booleans.

When to use

- Classify policies by prefix to drive product-specific assumptions.
- Identify riders with a particular prefix (e.g., primary benefits) when stored in a list column.
- Validate codes against expected prefixes coming from another column.

Parameters:

| Name     | Type  | Description | Default                                                                                                                                                                    |
| -------- | ----- | ----------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `prefix` | \`str | Expr\`      | The substring to check for at the beginning of each string. Can be a literal string (e.g., "TERM-") or a Polars expression (e.g., pl.col("another_column_with_prefixes")). |

Returns:

| Name              | Type              | Description                                                                                                                                                                         |
| ----------------- | ----------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `ExpressionProxy` | `ExpressionProxy` | A new ExpressionProxy containing a boolean Series indicating for each input string whether it starts with the prefix. If the input was List[String], the output will be List[bool]. |

Examples:

**Scalar example – policy prefixes**

```python
from gaspatchio.frame.base import ActuarialFrame

data_policies = {
    "policy_no": [
        "TERM-1001",
        "WL-2002",
        "TERM-1003",
        None,
        "UL-3004",
        "TERM-1004",
    ],
    "issue_age": [25, 30, 28, 45, 35, 40],
}
af = ActuarialFrame(data_policies)

# Check if policy_no starts with "TERM-"
af_term_policies = af.select(
    af["policy_no"].str.starts_with("TERM-").alias("is_term_policy")
)
print(af_term_policies.collect())
```

```text
shape: (6, 1)
┌────────────────┐
│ is_term_policy │
│ ---            │
│ bool           │
╞════════════════╡
│ true           │
│ false          │
│ true           │
│ null           │
│ false          │
│ true           │
└────────────────┘
```

**Vector (list) example – rider prefixes**

```python
from gaspatchio.frame.base import ActuarialFrame
import polars as pl

data_policy_riders = {
    "policy_id": ["P201", "P202", "P203"],
    "rider_codes_list": [
        ["B-ADB", "S-WP", "S-CI"],  # B-AccidentalDeathBenefit, S-WaiverOfPremium, S-CriticalIllness
        ["S-LTC", None, "B-GIO"],  # S-LongTermCare, B-GuaranteedInsurabilityOption
        ["S-WPR", "S-CIR"]
    ]
}
af_riders = ActuarialFrame(data_policy_riders).with_columns(
    pl.col("rider_codes_list").cast(pl.List(pl.String))
)

af_primary_benefit_check = af_riders.select(
    af_riders["rider_codes_list"].str.starts_with("B-").alias("has_primary_benefit_rider")
)
print(af_primary_benefit_check.collect())
```

```text
shape: (3, 1)
┌───────────────────────────┐
│ has_primary_benefit_rider │
│ ---                       │
│ list[bool]                │
╞═══════════════════════════╡
│ [true, false, false]      │
│ [false, null, true]       │
│ [false, false]            │
└───────────────────────────┘
```

### `strip_chars(characters=None)`

Removes specified leading and trailing characters from strings.

This is useful for cleaning data, such as removing unwanted prefixes, suffixes, or whitespace from policy numbers, client names, or address fields. It mirrors Polars' `Expr.str.strip_chars`. If no characters are specified, it defaults to removing leading and trailing whitespace. For `List[String]` columns, like a list of addresses for a client, the operation is applied element-wise to each string in the list.

When to use

- **Cleanse Identifier Fields:** Remove extraneous characters (e.g., spaces, hyphens, special symbols) from policy numbers, claim IDs, or client identifiers to ensure consistency for matching and lookups. For example, "POL- 123\* " could become "POL-123" by stripping " \*".
- **Standardize Textual Data:** Trim leading/trailing whitespace from free-text fields like occupation descriptions, addresses, or underwriter notes before analysis or storage.
- **Prepare Data for Joins:** Ensure that join keys consisting of string data are clean and consistently formatted to avoid join failures due to subtle differences like trailing spaces.
- **Sanitize User Input:** Clean user-provided search terms or filter values by removing unwanted characters before using them in queries.

Parameters:

| Name         | Type  | Description | Default                                                                                                                                                                                                         |
| ------------ | ----- | ----------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `characters` | \`str | Expr\`      | A string of characters to remove from both ends of each string. Can also be a Polars expression that evaluates to a string of characters. If None (default), removes whitespace (spaces, tabs, newlines, etc.). |

Returns:

| Name              | Type              | Description                                                                    |
| ----------------- | ----------------- | ------------------------------------------------------------------------------ |
| `ExpressionProxy` | `ExpressionProxy` | A new ExpressionProxy with the specified characters stripped from the strings. |

Examples:

**Scalar Example: Cleaning Policy Numbers**

Policy numbers might be recorded with inconsistent characters that need to be removed.

```python
from gaspatchio import ActuarialFrame

data = {
    "policy_id": ["P001", "P002", "P003", "P004"],
    "policy_number_raw": [" POL-123* ", "ID-456-", "*789*", " ABC-999 "],
}
af = ActuarialFrame(data)

af.policy_number_clean = af.policy_number_raw.str.strip_chars(" *-")

print(af.collect())
```

```text
shape: (4, 3)
┌───────────┬───────────────────┬─────────────────────┐
│ policy_id ┆ policy_number_raw ┆ policy_number_clean │
│ ---       ┆ ---               ┆ ---                 │
│ str       ┆ str               ┆ str                 │
╞═══════════╪═══════════════════╪═════════════════════╡
│ P001      ┆  POL-123*         ┆ POL-123             │
│ P002      ┆ ID-456-           ┆ ID-456              │
│ P003      ┆ *789*             ┆ 789                 │
│ P004      ┆  ABC-999          ┆ ABC-999             │
└───────────┴───────────────────┴─────────────────────┘
```

**Vector Example: Cleaning Lists of Rider Codes**

Product codes for riders might be stored in a list with unwanted characters.

```python
from gaspatchio import ActuarialFrame

data = {
    "policy_id": ["P1001", "P1002"],
    "rider_codes_raw": [
        ["*RIDER_A- ", " -RIDER_B*", "BASE_PLAN"],
        [None, " *-RIDER_C- ", "    RIDER_D *"]
    ]
}
af = ActuarialFrame(data)

af.rider_codes_clean = af.rider_codes_raw.str.strip_chars(" *-      ")

print(af.collect())
```

```text
shape: (2, 3)
┌───────────┬───────────────────────────────────────────┬─────────────────────────────────────┐
│ policy_id ┆ rider_codes_raw                           ┆ rider_codes_clean                   │
│ ---       ┆ ---                                       ┆ ---                                 │
│ str       ┆ list[str]                                 ┆ list[str]                           │
╞═══════════╪═══════════════════════════════════════════╪═════════════════════════════════════╡
│ P1001     ┆ ["*RIDER_A- ", " -RIDER_B*", "BASE_PLAN"] ┆ ["RIDER_A", "RIDER_B", "BASE_PLAN"] │
│ P1002     ┆ [null, " *-RIDER_C- ", "    RIDER_D *"]   ┆ [null, "RIDER_C", "RIDER_D"]        │
└───────────┴───────────────────────────────────────────┴─────────────────────────────────────┘
```

### `strip_chars_start(characters=None)`

Removes specified leading characters from strings.

Useful for standardizing data by removing known prefixes or initial whitespace. For instance, cleaning policy numbers by removing a "TEMP-" prefix or trimming spaces from the beginning of address lines. It mirrors Polars' `Expr.str.strip_chars_start`. If no characters are specified, it defaults to removing leading whitespace. When applied to `List[String]` columns (e.g., a list of historical status codes for a policy), the operation is performed element-wise.

When to use

- **Normalizing Prefixed Identifiers:** Removing consistent prefixes from identifiers like policy numbers (e.g., "PN-", "TEMP\_"), claim codes (e.g., "CL-"), or agent codes to get the core identifier.
- **Cleaning Leading Characters in Text Fields:** Removing leading non-essential characters (e.g., bullets, numbers, special symbols, spaces) from free-text fields like notes, descriptions, or imported data before further processing.
- **Standardizing Data from Multiple Sources:** If different source systems prefix the same data differently, this function can help unify them by removing those specific leading characters.

Parameters:

| Name         | Type  | Description | Default                                                                                                                                                                                                                 |
| ------------ | ----- | ----------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `characters` | \`str | Expr\`      | A string of characters to remove from the start of each string. Can also be a Polars expression that evaluates to a string of characters. If None (default), removes leading whitespace (spaces, tabs, newlines, etc.). |

Returns:

| Name              | Type              | Description                                                                        |
| ----------------- | ----------------- | ---------------------------------------------------------------------------------- |
| `ExpressionProxy` | `ExpressionProxy` | A new ExpressionProxy with specified leading characters stripped from the strings. |

Examples:

**Scalar Example: Removing Leading Characters from Policy IDs**

Legacy system IDs might have prefixes or leading whitespace that need cleaning.

```python
from gaspatchio import ActuarialFrame

data = {
    "policy_id": ["P001", "P002", "P003", "P004"],
    "legacy_id": ["TEMP-123", "  456", " *789", "TEMP-ABC"],
}
af = ActuarialFrame(data)

af.clean_id = af.legacy_id.str.strip_chars_start(" TEMP-*")

print(af.collect())
```

```text
shape: (4, 3)
┌───────────┬───────────┬──────────┐
│ policy_id ┆ legacy_id ┆ clean_id │
│ ---       ┆ ---       ┆ ---      │
│ str       ┆ str       ┆ str      │
╞═══════════╪═══════════╪══════════╡
│ P001      ┆ TEMP-123  ┆ 123      │
│ P002      ┆   456     ┆ 456      │
│ P003      ┆  *789     ┆ 789      │
│ P004      ┆ TEMP-ABC  ┆ ABC      │
└───────────┴───────────┴──────────┘
```

**Vector Example: Transaction Remarks**

```python
from gaspatchio import ActuarialFrame

data = {
    "policy_id": ["P001", "P002"],
    "remarks": [
        ["TEMP: Initial assessment", "  Adjustment processed", "Final Review"],
        ["TEMP: Hold for now", "TEMP: Resolved", "Status: OK"],
    ],
}
af = ActuarialFrame(data)

af.clean_remarks = af.remarks.str.strip_chars_start("TEMP: ")

print(af.collect())
```

```text
shape: (2, 3)
┌───────────┬────────────────────────────────────────────────────────────────────────┬────────────────────────────────────────────────────────────────┐
│ policy_id ┆ remarks                                                                ┆ clean_remarks                                                  │
│ ---       ┆ ---                                                                    ┆ ---                                                            │
│ str       ┆ list[str]                                                              ┆ list[str]                                                      │
╞═══════════╪════════════════════════════════════════════════════════════════════════╪════════════════════════════════════════════════════════════════╡
│ P001      ┆ ["TEMP: Initial assessment", "  Adjustment processed", "Final Review"] ┆ ["Initial assessment", "Adjustment processed", "Final Review"] │
│ P002      ┆ ["TEMP: Hold for now", "TEMP: Resolved", "Status: OK"]                 ┆ ["Hold for now", "Resolved", "Status: OK"]                     │
└───────────┴────────────────────────────────────────────────────────────────────────┴────────────────────────────────────────────────────────────────┘
```

### `strip_prefix(prefix)`

Remove a prefix from each string.

The prefix is stripped whenever it occurs at the start of the string. Strings without the prefix are returned unchanged. On columns containing lists of strings, the removal happens element by element.

When to use

- Cleaning temporary identifiers such as `TEMP-123` once a policy is fully underwritten.
- Harmonizing product codes from different administration systems before mapping them to an actuarial model.
- Stripping `LEGACY-` markers from lists of rider codes imported from historical sources.

Parameters:

| Name     | Type  | Description | Default                                                                                |
| -------- | ----- | ----------- | -------------------------------------------------------------------------------------- |
| `prefix` | \`str | Expr\`      | Prefix to remove. May be a literal string or an expression that evaluates to a string. |

Returns:

| Type              | Description                              |
| ----------------- | ---------------------------------------- |
| `ExpressionProxy` | ExpressionProxy with the prefix removed. |

Examples:

**Scalar Example: Policy IDs**

```python
from gaspatchio import ActuarialFrame

data = {
    "policy_id": ["P001", "P002", "P003", "P004"],
    "pol_id_raw": ["TEMP-001", "TEMP-002", "003", None],
}
af = ActuarialFrame(data)

af.pol_id_clean = af.pol_id_raw.str.strip_prefix("TEMP-")

print(af.collect())
```

```text
shape: (4, 3)
┌───────────┬────────────┬──────────────┐
│ policy_id ┆ pol_id_raw ┆ pol_id_clean │
│ ---       ┆ ---        ┆ ---          │
│ str       ┆ str        ┆ str          │
╞═══════════╪════════════╪══════════════╡
│ P001      ┆ TEMP-001   ┆ 001          │
│ P002      ┆ TEMP-002   ┆ 002          │
│ P003      ┆ 003        ┆ 003          │
│ P004      ┆ null       ┆ null         │
└───────────┴────────────┴──────────────┘
```

**Vector Example: Feature Codes**

```python
from gaspatchio import ActuarialFrame

data = {
    "policy_id": ["P001", "P002"],
    "feature_codes": [
        ["LEGACY-RIDER1", "NEW_FEATURE_X", "LEGACY-BENEFIT2"],
        ["LEGACY-COVERAGE_Y", "STANDARD_Z"],
    ],
}
af = ActuarialFrame(data)

af.clean_codes = af.feature_codes.str.strip_prefix("LEGACY-")

print(af.collect())
```

```text
shape: (2, 3)
┌───────────┬───────────────────────────────────────────────────────┬─────────────────────────────────────────┐
│ policy_id ┆ feature_codes                                         ┆ clean_codes                             │
│ ---       ┆ ---                                                   ┆ ---                                     │
│ str       ┆ list[str]                                             ┆ list[str]                               │
╞═══════════╪═══════════════════════════════════════════════════════╪═════════════════════════════════════════╡
│ P001      ┆ ["LEGACY-RIDER1", "NEW_FEATURE_X", "LEGACY-BENEFIT2"] ┆ ["RIDER1", "NEW_FEATURE_X", "BENEFIT2"] │
│ P002      ┆ ["LEGACY-COVERAGE_Y", "STANDARD_Z"]                   ┆ ["COVERAGE_Y", "STANDARD_Z"]            │
└───────────┴───────────────────────────────────────────────────────┴─────────────────────────────────────────┘
```

### `strip_suffix(suffix)`

Remove a suffix from each string.

If a string does not end with the given suffix, it is returned unchanged. For `List[String]` columns, the operation is applied element-wise.

When to use

- **Normalizing coverage names** that include trailing version codes such as "-OLD".
- **Preparing ledger accounts** by removing year suffixes like "-2024" before comparing periods.
- **Cleaning temporary identifiers** imported from external systems (for example, removing a trailing "-TMP").

Parameters:

| Name     | Type  | Description | Default                                                                               |
| -------- | ----- | ----------- | ------------------------------------------------------------------------------------- |
| `suffix` | \`str | Expr\`      | The suffix to remove. Either a string literal or an expression resolving to a string. |

Returns:

| Name              | Type              | Description                             |
| ----------------- | ----------------- | --------------------------------------- |
| `ExpressionProxy` | `ExpressionProxy` | The expression with the suffix removed. |

Examples:

**Scalar Example: Plan Names**

```python
from gaspatchio import ActuarialFrame

data = {
    "policy_id": ["P001", "P002", "P003", "P004"],
    "plan_name_raw": [
        "Term Basic-OLD",
        "Income Protection-OLD",
        "Annuity Plus",
        None,
    ],
}
af = ActuarialFrame(data)

af.plan_name = af.plan_name_raw.str.strip_suffix("-OLD")

print(af.collect())
```

```text
shape: (4, 3)
┌───────────┬───────────────────────┬───────────────────┐
│ policy_id ┆ plan_name_raw         ┆ plan_name         │
│ ---       ┆ ---                   ┆ ---               │
│ str       ┆ str                   ┆ str               │
╞═══════════╪═══════════════════════╪═══════════════════╡
│ P001      ┆ Term Basic-OLD        ┆ Term Basic        │
│ P002      ┆ Income Protection-OLD ┆ Income Protection │
│ P003      ┆ Annuity Plus          ┆ Annuity Plus      │
│ P004      ┆ null                  ┆ null              │
└───────────┴───────────────────────┴───────────────────┘
```

**Vector Example: Claim Notes**

```python
from gaspatchio import ActuarialFrame

data = {
    "policy_id": ["P001", "P002"],
    "notes": [
        ["Approved.", "Paid."],
        ["In Review."],
    ],
}
af = ActuarialFrame(data)

af.notes_clean = af.notes.str.strip_suffix(".")

print(af.collect())
```

```text
shape: (2, 3)
┌───────────┬────────────────────────┬──────────────────────┐
│ policy_id ┆ notes                  ┆ notes_clean          │
│ ---       ┆ ---                    ┆ ---                  │
│ str       ┆ list[str]              ┆ list[str]            │
╞═══════════╪════════════════════════╪══════════════════════╡
│ P001      ┆ ["Approved.", "Paid."] ┆ ["Approved", "Paid"] │
│ P002      ┆ ["In Review."]         ┆ ["In Review"]        │
└───────────┴────────────────────────┴──────────────────────┘
```

### `strptime(dtype, format=None, *, strict=True, exact=True, cache=True, ambiguous='raise', **kwargs)`

Convert string values to Date, Datetime, or Time.

This method parses textual date or time information into Polars temporal types. For `List[String]` columns, each element is parsed individually.

When to use

- Convert policy issue or claim reporting dates that are stored as strings in raw data extracts.
- Parse lists of event timestamps—such as claim status updates—when building experience studies or exposure models.
- Ingest external datasets from underwriting or administration systems where date fields come in a variety of text formats.

Parameters:

| Name        | Type                 | Description                                                                | Default                                                                                                                                                                  |
| ----------- | -------------------- | -------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `dtype`     | `PolarsTemporalType` | The Polars temporal type to convert to (pl.Date, pl.Datetime, or pl.Time). | *required*                                                                                                                                                               |
| `format`    | \`str                | None\`                                                                     | The strf/strptime format string. If None, the format is inferred where possible.                                                                                         |
| `strict`    | `bool`               | If True (default), raise an error on parsing failure.                      | `True`                                                                                                                                                                   |
| `exact`     | `bool`               | If True (default), require an exact format match.                          | `True`                                                                                                                                                                   |
| `cache`     | `bool`               | If True (default), cache parsing results for performance.                  | `True`                                                                                                                                                                   |
| `ambiguous` | \`str                | Expr\`                                                                     | How to handle ambiguous datetimes, such as daylight-saving transitions. Options are "raise" (default), "earliest", "latest", or "null". Can also be a Polars expression. |

Returns:

| Name              | Type              | Description                                       |
| ----------------- | ----------------- | ------------------------------------------------- |
| `ExpressionProxy` | `ExpressionProxy` | Strings converted to the specified temporal type. |

Examples:

**Scalar Example: Parsing policy issue dates**

```python
from gaspatchio.frame.base import ActuarialFrame
import polars as pl

data = {
    "policy_id": ["A100", "B200", "C300"],
    "issue_date_str": ["2021-01-15", "20/02/2022", "2023-03-10 14:30:00"],
}
af = ActuarialFrame(data)
af_parsed_dates = af.select(
    af["issue_date_str"]
    .str.strptime(pl.Date, "%Y-%m-%d", strict=False)
    .alias("issue_date_strict_fmt"),
    af["issue_date_str"]
    .str.strptime(pl.Date, "%d/%m/%Y", strict=False)
    .alias("issue_date_dmy_fmt"),
    af["issue_date_str"]
    .str.strptime(pl.Datetime, "%Y-%m-%d %H:%M:%S", strict=False)
    .alias("issue_datetime"),
)
result = af_parsed_dates.collect()
print(result)
```

```text
shape: (3, 3)
┌───────────────────────┬────────────────────┬─────────────────────┐
│ issue_date_strict_fmt ┆ issue_date_dmy_fmt ┆ issue_datetime      │
│ ---                   ┆ ---                ┆ ---                 │
│ date                  ┆ date               ┆ datetime[μs]        │
╞═══════════════════════╪════════════════════╪═════════════════════╡
│ 2021-01-15            ┆ null               ┆ null                │
│ null                  ┆ 2022-02-20         ┆ null                │
│ null                  ┆ null               ┆ 2023-03-10 14:30:00 │
└───────────────────────┴────────────────────┴─────────────────────┘
```

**Vector Example: Parsing lists of event timestamps**

```python
from gaspatchio.frame.base import ActuarialFrame
import polars as pl

data_list = {
    "claim_id": ["CL001"],
    "event_timestamps_str": [["2023-04-01T10:00:00", "2023-04-01T10:05:00", "Invalid"]],
}
af_list = ActuarialFrame(data_list).with_columns(
    pl.col("event_timestamps_str").cast(pl.List(pl.String))
)
af_parsed_list = af_list.select(
    af_list["event_timestamps_str"].str.strptime(
        pl.Datetime, "%Y-%m-%dT%H:%M:%S", strict=False
    ).alias("event_datetimes_μs")
)
result = af_parsed_list.collect()
print(result)
```

```text
shape: (1, 1)
┌──────────────────────────────────────────────────┐
│ event_datetimes_μs                               │
│ ---                                              │
│ list[datetime[μs]]                               │
╞══════════════════════════════════════════════════╡
│ [2023-04-01 10:00:00, 2023-04-01 10:05:00, null] │
└──────────────────────────────────────────────────┘
```

### `to_lowercase()`

Converts all characters in string columns to lowercase.

This function standardizes textual data by converting all characters in a string column to lowercase. This is essential for ensuring consistency in data fields critical for actuarial analysis, such as system codes, free-text fields like occupation or medical conditions, or external data sources, facilitating accurate matching, aggregation, and text analysis.

When to use

- **Normalizing Text for Analysis:** Preparing free-text fields (e.g., underwriting notes, claim descriptions, occupation details) for text mining or NLP by ensuring terms like "SMOKER", "Smoker", and "smoker" are treated identically.
- **Improving Data Matching with External Sources:** When integrating data from various systems or third-party providers where case consistency is not guaranteed (e.g., matching addresses, names, or city information).
- **Standardizing User Input:** Converting user-entered data (e.g., search terms, filter criteria) to a consistent case before processing or querying.

Returns:

| Name              | Type              | Description                                             |
| ----------------- | ----------------- | ------------------------------------------------------- |
| `ExpressionProxy` | `ExpressionProxy` | An ExpressionProxy with strings converted to lowercase. |

Examples:

**Scalar Example: Normalizing occupation descriptions for risk analysis**

Occupation descriptions might be entered in various casings. Converting to lowercase helps in standardizing them for consistent risk factor analysis or grouping.

```python
from gaspatchio.frame.base import ActuarialFrame

data = {
    "policy_id": ["POL001", "POL002", "POL003", "POL004"],
    "occupation_raw": [
        "Engineer",
        "software DEVELOPER",
        "Teacher",
        "Project Manager",
    ],
}
af = ActuarialFrame(data)
af_lower_occupation = af.select(
    af["occupation_raw"].str.to_lowercase().alias("occupation_normalized")
)
print(af_lower_occupation.collect())
```

```text
shape: (4, 1)
┌───────────────────────┐
│ occupation_normalized │
│ ---                   │
│ str                   │
╞═══════════════════════╡
│ engineer              │
│ software developer    │
│ teacher               │
│ project manager       │
└───────────────────────┘
```

**Vector Example: Lowercasing medical condition codes from multiple sources**

Medical condition codes might come from different systems with varying casing. Lowercasing them ensures they can be consistently mapped or analyzed.

```python
from gaspatchio import ActuarialFrame

data = {
    "claim_id": ["C001", "C002"],
    "condition_codes": [
        ["DIAB_T2", "HBP", "ASTHMA"],
        ["hbp", None, "COPD"]
    ]
}
af = ActuarialFrame(data)

af.lower_codes = af.condition_codes.str.to_lowercase()

print(af.collect())
```

```text
shape: (2, 3)
┌──────────┬──────────────────────────────┬──────────────────────────────┐
│ claim_id ┆ condition_codes              ┆ lower_codes                  │
│ ---      ┆ ---                          ┆ ---                          │
│ str      ┆ list[str]                    ┆ list[str]                    │
╞══════════╪══════════════════════════════╪══════════════════════════════╡
│ C001     ┆ ["DIAB_T2", "HBP", "ASTHMA"] ┆ ["diab_t2", "hbp", "asthma"] │
│ C002     ┆ ["hbp", null, "COPD"]        ┆ ["hbp", null, "copd"]        │
└──────────┴──────────────────────────────┴──────────────────────────────┘
```

### `to_uppercase()`

Converts all characters in string columns to uppercase.

This function standardizes textual data by converting all characters in a string column to uppercase. This is essential for ensuring consistency in data fields critical for actuarial analysis, such as policy status codes, product identifiers, or geographical regions, facilitating accurate matching, aggregation, and reporting.

When to use

- **Standardizing Categorical Data:** Ensuring that codes like policy status (e.g., "active", "Lapsed", "ACTIVE" all become "ACTIVE"), gender codes (e.g., "m", "F" become "M", "F"), or smoker status (e.g. "non-smoker", "Smoker" become "NON-SMOKER", "SMOKER") are consistent for grouping and analysis.
- **Improving Data Matching:** Facilitating joins and lookups between different datasets where case sensitivity might cause mismatches (e.g., matching policyholder names or addresses from different sources).
- **Enhancing Readability and Reporting:** Presenting data in a uniform case for reports and dashboards, especially for identifiers or codes.
- **Preparing Text for Analysis:** As a preprocessing step before text mining or natural language processing tasks on fields like claim descriptions or underwriter notes, where case normalization can simplify pattern recognition.
- **Simplifying Rule-Based Logic:** When applying business rules that depend on string comparisons (e.g., identifying policies with specific rider codes like "ADB" or "WP" irrespective of their original casing).

Returns:

| Name              | Type              | Description                                                |
| ----------------- | ----------------- | ---------------------------------------------------------- |
| `ExpressionProxy` | `ExpressionProxy` | A new ExpressionProxy with strings converted to uppercase. |

Examples:

**Scalar Example: Standardizing policy status codes**

Policy status might be entered in various cases ("active", "lapsed", "ACTIVE"). Converting to uppercase ensures consistency for analysis.

```python
from gaspatchio.frame.base import ActuarialFrame

data = {
    "policy_id": ["S3001", "S3002", "S3003", "S3004"],
    "status_raw": ["active", "lapsed", "Active", "PENDING"],
}
af = ActuarialFrame(data)
af_upper_status = af.select(
    af["status_raw"].str.to_uppercase().alias("status_standardized")
)
print(af_upper_status.collect())
```

```text
shape: (4, 1)
┌─────────────────────┐
│ status_standardized │
│ ---                 │
│ str                 │
╞═════════════════════╡
│ ACTIVE              │
│ LAPSED              │
│ ACTIVE              │
│ PENDING             │
└─────────────────────┘
```

**Vector Example: Uppercasing rider codes for a policy**

A policy might have multiple rider codes stored in a list. To ensure uniformity, we can convert all rider codes to uppercase.

```python
from gaspatchio.frame.base import ActuarialFrame

data_policy_riders = {
    "policy_id": ["R4001", "R4002", "R4003"],
    "rider_codes_str": [
        "adb,wp",
        "ci,ltc,acc_death",
        "gio"
    ]
}
af_riders = ActuarialFrame(data_policy_riders)
# Convert string to list for the string operation
af_riders = af_riders.with_columns(
    af_riders["rider_codes_str"].str.split(",").alias("rider_codes_list")
)
af_upper_riders = af_riders.select(
    af_riders["rider_codes_list"].str.to_uppercase().alias("upper_rider_codes")
)
print(af_upper_riders.collect())
```

```text
shape: (3, 1)
┌────────────────────────────┐
│ upper_rider_codes          │
│ ---                        │
│ list[str]                  │
╞════════════════════════════╡
│ ["ADB", "WP"]              │
│ ["CI", "LTC", "ACC_DEATH"] │
│ ["GIO"]                    │
└────────────────────────────┘
```

### `zfill(length)`

Pad strings with leading zeros to a minimum width.

Shorter values are padded on the left with zeros so each entry reaches `length` characters. For list columns, the padding occurs element-wise.

When to use

- Standardizing policy numbers from different administration systems before merging with valuation data
- Preparing zero-padded claim numbers for extracts sent to reinsurers or regulators
- Building fixed-width keys when joining to rating tables or mapping grids

Parameters:

| Name     | Type  | Description                               | Default    |
| -------- | ----- | ----------------------------------------- | ---------- |
| `length` | `int` | The desired minimum length of the string. | *required* |

Returns:

| Name              | Type              | Description                        |
| ----------------- | ----------------- | ---------------------------------- |
| `ExpressionProxy` | `ExpressionProxy` | Strings padded with leading zeros. |

Examples:

**Scalar Example: Policy Serial Numbers**

```python
from gaspatchio import ActuarialFrame

data = {
    "policy_id": ["P001", "P002", "P003", "P004", "P005"],
    "policy_serial": ["123", "45", "6789", None, "1"],
}
af = ActuarialFrame(data)

af.zfilled_serial = af.policy_serial.str.zfill(5)

print(af.collect())
```

```text
shape: (5, 3)
┌───────────┬───────────────┬────────────────┐
│ policy_id ┆ policy_serial ┆ zfilled_serial │
│ ---       ┆ ---           ┆ ---            │
│ str       ┆ str           ┆ str            │
╞═══════════╪═══════════════╪════════════════╡
│ P001      ┆ 123           ┆ 00123          │
│ P002      ┆ 45            ┆ 00045          │
│ P003      ┆ 6789          ┆ 06789          │
│ P004      ┆ null          ┆ null           │
│ P005      ┆ 1             ┆ 00001          │
└───────────┴───────────────┴────────────────┘
```

**Vector Example: Claim Item Codes**

```python
from gaspatchio import ActuarialFrame

data = {
    "policy_id": ["P001", "P002"],
    "item_codes": [
        ["A1", "B123", "C04"],
        ["D56"],
    ],
}
af = ActuarialFrame(data)

af.zfilled_codes = af.item_codes.str.zfill(4)

print(af.collect())
```

```text
shape: (2, 3)
┌───────────┬───────────────────────┬──────────────────────────┐
│ policy_id ┆ item_codes            ┆ zfilled_codes            │
│ ---       ┆ ---                   ┆ ---                      │
│ str       ┆ list[str]             ┆ list[str]                │
╞═══════════╪═══════════════════════╪══════════════════════════╡
│ P001      ┆ ["A1", "B123", "C04"] ┆ ["00A1", "B123", "0C04"] │
│ P002      ┆ ["D56"]               ┆ ["0D56"]                 │
└───────────┴───────────────────────┴──────────────────────────┘
```

## `gaspatchio.accessors.excel_functions.days.days(end_date, start_date)`

Calculate the number of days between two dates, similar to Excel's DAYS.

Returns the number of days between a start date and an end date as an integer. The result is positive if the end date is after the start date, and negative if the end date is before the start date.

When to use

- **Policy Duration Calculations:** Determine the exact number of days a policy has been in force for premium calculations or exposure analysis.
- **Claim Processing Time:** Calculate the number of days between claim filing and settlement for service level tracking.
- **Grace Period Tracking:** Measure the number of days in grace periods for lapsed policies or late premium payments.
- **Interest Accrual:** Calculate the exact number of days for interest calculations on policy loans or reserves.
- **Waiting Period Compliance:** Track waiting periods in days for specific benefits or coverage exclusions.
- **Performance Metrics:** Measure time-to-issue, underwriting duration, or other process timelines in days.

#### Parameters

end_date : IntoExprColumn The ending date of the period. Can be a scalar date, a column of dates, or a list column of dates. start_date : IntoExprColumn The starting date of the period. Can be a scalar date, a column of dates, or a list column of dates.

#### Returns

pl.Expr A Polars expression containing the number of days as Int64 (or List[Int64] for list columns). The result is end_date - start_date.

#### Examples

**Scalar Example: Policy Duration in Days**

```python
import datetime
from gaspatchio import ActuarialFrame

data = {
    "policy_id": ["P001", "P002", "P003"],
    "issue_date": [
        datetime.date(2023, 1, 15),
        datetime.date(2023, 3, 1),
        datetime.date(2023, 6, 10)
    ],
    "valuation_date": [
        datetime.date(2023, 12, 31),
        datetime.date(2023, 12, 31),
        datetime.date(2023, 12, 31)
    ],
}
af = ActuarialFrame(data)

af.days_in_force = af.valuation_date.excel.days(af.issue_date)

print(af.collect())
```

```text
shape: (3, 4)
┌───────────┬────────────┬────────────────┬───────────────┐
│ policy_id ┆ issue_date ┆ valuation_date ┆ days_in_force │
│ ---       ┆ ---        ┆ ---            ┆ ---           │
│ str       ┆ date       ┆ date           ┆ i64           │
╞═══════════╪════════════╪════════════════╪═══════════════╡
│ P001      ┆ 2023-01-15 ┆ 2023-12-31     ┆ 350           │
│ P002      ┆ 2023-03-01 ┆ 2023-12-31     ┆ 305           │
│ P003      ┆ 2023-06-10 ┆ 2023-12-31     ┆ 204           │
└───────────┴────────────┴────────────────┴───────────────┘
```

**Vector Example: Monthly Projection Days**

```python
import datetime
import polars as pl
from gaspatchio import ActuarialFrame
from gaspatchio.accessors.excel_functions.days import days

data = {
    "policy_id": ["P001", "P002"],
    "projection_dates": [
        [datetime.date(2024, 1, 1), datetime.date(2024, 2, 1), datetime.date(2024, 3, 1)],
        [datetime.date(2024, 1, 15), datetime.date(2024, 2, 15), datetime.date(2024, 3, 15)]
    ],
    "issue_date": [datetime.date(2024, 1, 1), datetime.date(2024, 1, 1)]
}
af = ActuarialFrame(data)

af.days_from_issue = af.projection_dates.list.eval(
    days(pl.element(), pl.lit(datetime.date(2024, 1, 1)))
)

print(af.collect())
```

```text
shape: (2, 4)
┌───────────┬──────────────────────────────────────┬────────────┬─────────────────┐
│ policy_id ┆ projection_dates                     ┆ issue_date ┆ days_from_issue │
│ ---       ┆ ---                                  ┆ ---        ┆ ---             │
│ str       ┆ list[date]                           ┆ date       ┆ list[i64]       │
╞═══════════╪══════════════════════════════════════╪════════════╪═════════════════╡
│ P001      ┆ [2024-01-01, 2024-02-01, 2024-03-01] ┆ 2024-01-01 ┆ [0, 31, 60]     │
│ P002      ┆ [2024-01-15, 2024-02-15, 2024-03-15] ┆ 2024-01-01 ┆ [14, 45, 74]    │
└───────────┴──────────────────────────────────────┴────────────┴─────────────────┘
```