Welcome to the World of Graduation!

In actuarial work, we often deal with raw mortality or transition data. When we observe raw (crude) rates, they fluctuate from age to age due to sampling noise, small exposures, or chance events. In reality, we expect underlying biological mortality to progress smoothly with age.

Graduation is the process of smoothing these bumpy crude rates to produce a set of transition intensities or probabilities that are regular, reliable, and suitable for pricing, reserving, and financial projections.

In this chapter, we explore why we graduate data, the three syllabus methods of graduation, how to test for smoothness, and the suite of statistical tests used to evaluate adherence to data and detect bias.

1. Why do we Graduate?

If our crude sample shows that 50-year-olds have twice the mortality rate of 51-year-olds purely because of accidental deaths in a modest sample, pricing contracts on crude rates would lead to erratic premiums and commercial distortions.

We graduate crude estimates for three primary reasons:

1. To remove random sampling fluctuations: Raw data contains noise that obscures the true underlying pattern.
2. To ensure regularity and smoothness: Rates should progress logically across ages (e.g. mortality generally rises across adult ages) so that calculated premiums and reserves progress regularly.
3. To estimate rates at sparse ages: Smoothing allows interpolation for ages where observed data is thin or missing.

The Great Balancing Act: Fit vs. Smoothness

Graduation involves a fundamental trade-off between two competing criteria:

• Adherence to data (Goodness of Fit): The graduated curve should stay close to the observed crude rates without distorting the underlying level.
• Smoothness: The progression of rates across adjacent ages should be gradual and free of irregular kinks or sharp oscillations.

Analogy: Joining every observed dot with straight segments produces 100% adherence but zero smoothness. A flat horizontal line has 100% smoothness but terrible fit. A successful graduation finds the optimal balance between the two.

2. Methods of Graduation

The CS2 syllabus specifies three primary graduation approaches. Note: Under syllabus objective 4.5, candidates will not be required to carry out a graduation in the examination, but must know the processes, principles, and advantages/disadvantages of each method.

A. Parametric Formula (Mathematical Formula)

We fit a mathematical curve to the crude transition intensities or mortality rates across the entire age range, such as Gompertz's law ( \( \mu_x = B c^x \) ), Makeham's law ( \( \mu_x = A + B c^x \) ), or polynomial formulas.
• Advantages: Produces inherently smooth rates; compact to store and program; easy to calculate actuarial functions.
• Disadvantages: Inflexible if the population's true mortality shape deviates from the chosen formula over certain age bands.

B. Reference to a Standard Table

We adjust an established, widely accepted standard mortality table to fit our specific experience using a simple mathematical link function, such as a linear transformation ( \( \mathring{q}_x = a + b q_x^s \) ) or a logit transformation ( \( \text{logit}(\mathring{q}_x) = \alpha + \beta \, \text{logit}(q_x^s) \) ).
• Advantages: Highly effective for small datasets where crude data alone cannot determine the shape of the mortality curve; borrows realistic biological shape from a credible population.
• Disadvantages: Constrained by the shape of the chosen standard table; unsuitable if no appropriate standard table exists.

C. Spline Functions

Splines divide the age range into segments separated by boundary points called knots, fitting low-order polynomials (typically cubic splines) to each segment while constraining the curve and its first and second derivatives to be continuous at the knots.
• Advantages: Highly flexible; can fit complex, non-standard mortality patterns locally without causing unwanted ripples in other age ranges.
• Disadvantages: Selecting the number and placement of knots can be subjective; placing too many knots leads to overfitting (reproducing random noise).

3. Testing for Smoothness

Smoothness is a property of the graduated rates themselves, independent of the observed data.

• Finite Differences: The standard method to test smoothness is by calculating successive differences of the graduated rates, particularly the third differences ( \( \Delta^3 \mathring{q}_x \) ).
• Criteria for Smoothness: Third differences should be small in magnitude, progress smoothly across ages, and avoid frequent alternating signs. A sequence of third differences that jumps erratically indicates lack of smoothness.

4. Statistical Tests for Goodness of Fit and Bias

Once rates are graduated, we test whether the graduated rates ( \( \mathring{q}_x \) or \( \mathring{\mu}_x \) ) adhere adequately to the crude data, have no systematic bias, and follow the true underlying shape. Let \( d_x \) be observed deaths, \( E_x \) the initial exposed to risk, and \( \hat{d}_x = E_x \mathring{q}_x \) the expected deaths under the graduated rates.

The standardised deviation (residual) at age \( x \) is:
\( z_x = \frac{d_x - E_x \mathring{q}_x}{\sqrt{E_x \mathring{q}_x (1 - \mathring{q}_x)}} \approx \frac{d_x - \hat{d}_x}{\sqrt{\text{Var}(d_x)}} \)

A. Chi-Square ( \( \chi^2 \) ) Test (Overall Goodness of Fit)

Tests overall adherence across all \( m \) ages.
• Test Statistic: \( \sum_{x} z_x^2 = \sum_{x} \frac{(d_x - E_x \mathring{q}_x)^2}{E_x \mathring{q}_x (1 - \mathring{q}_x)} \sim \chi_k^2 \)
• Degrees of Freedom: \( k = m - p \) , where \( m \) is the number of age groups and \( p \) is the number of parameters estimated during graduation (when tested against a standard table with no parameters estimated from the data, \( p = 0 \) ).
• Application: If the statistic exceeds the upper critical value of \( \chi_k^2 \) , reject \( H_0 \) (poor overall fit).

B. Standardised Deviations Test (Individual Outliers)

Tests whether individual deviations conform to standard normal behaviour \( z_x \sim N(0, 1) \) .
• Application: We check whether roughly 68% of \( z_x \) fall in \( (-1, 1) \) , 95% in \( (-2, 2) \) , and virtually all in \( (-3, 3) \) . A single \( |z_x| > 3 \) indicates an isolated age with significant poor fit.

C. Cumulative Deviations Test (Overall Bias)

Tests for overall bias across the entire age range (whether the graduated rates systematically over- or under-estimate total deaths).
• Test Statistic: \( \frac{\sum (d_x - \hat{d}_x)}{\sqrt{\sum \text{Var}(d_x)}} \sim N(0, 1) \)
• Application: Two-tailed normal test. A large positive or negative value indicates that the graduation is consistently biased.

D. Signs Test (Frequency of Directional Bias)

Counts the number of positive deviations \( P \) out of \( m \) ages.
• Distribution: Under \( H_0 \) (equal probability of over- or under-estimating), \( P \sim \text{Binomial}(m, 0.5) \) (or a normal approximation for large \( m \) ).
• Application: Detects whether one sign occurs significantly more often than expected by chance.

E. Grouping of Signs Test (Runs Test for Local Bias)

Tests whether positive and negative signs clump together into extended runs, which indicates that the graduated curve cuts across the data and fails to capture the true local shape.
• Application: Count the number of positive runs \( G \) . Compare \( G \) to critical values from the Steven-Dodd distribution or normal approximation. Too few runs indicates local bias (clumping).

F. Serial Correlation Test (Consistency of Shape / Autocorrelation)

Tests whether adjacent residuals are correlated, which reveals whether the graduated curve systematically twists around the crude data.
• Test Statistic: The sample lag-1 autocorrelation coefficient \( r_1 = \frac{\sum_{j=1}^{m-1} (z_j - \bar{z})(z_{j+1} - \bar{z})}{\sum_{j=1}^m (z_j - \bar{z})^2} \)
• Distribution: For large \( m \) , \( r_1 \sqrt{m} \sim N(0, 1) \) .
• Application: A significantly positive \( r_1 \) indicates positive serial correlation (deviations at age \( x+1 \) mirror age \( x \) ).

5. Adjusting Fit Tests for Duplicate Policies

In life insurance and pension datasets, a single individual may hold multiple policies. If an individual dies, multiple death claims occur simultaneously, violating the assumption of mutual independence between policy deaths.

• Impact on Variance: While the expected number of deaths remains unchanged ( \( E[d_x] = \hat{d}_x \) ), the variance of observed deaths is inflated by a factor \( r_x > 1 \) :
\( \text{Var}(d_x) = r_x \cdot E_x \mathring{q}_x (1 - \mathring{q}_x) \)
where \( r_x = \frac{\sum k^2 n_k}{\sum k n_k} \) is the ratio of the second moment to the first moment of the policy count distribution per life.
• Adjustment to Test Statistics: If duplicate policies are ignored, test statistics like the Chi-square and individual standardised deviations will be artificially inflated, leading to false rejections of valid graduations. To amend the tests, divide \( \chi^2 \) by \( r \) , and divide standardised deviations by \( \sqrt{r} \) .

6. Summary of Tests at a Glance

1. Third Differences: Tests smoothness of the graduated curve itself.
2. Chi-Square Test: Tests overall goodness of fit across all ages (allowing for degrees of freedom lost).
3. Cumulative Deviations: Tests for net overall directional bias.
4. Signs Test: Tests for directional bias in the proportion of positive vs. negative residuals.
5. Grouping of Signs: Tests for runs of like signs, detecting local shape distortion.
6. Serial Correlation: Tests for lag-1 correlation between adjacent deviations.
7. Standardised Deviations: Tests for isolated ages of unacceptable fit (outliers).