Welcome to the World of Multivariate Moments!

Hi there! You’ve already learned how to find the mean (expected value) and variance for a single variable. Now, we are stepping into the multivariate setting. Think of this as moving from looking at a single policyholder's claim count to looking at the joint distribution of multiple discrete variables, such as claim frequency and policyholder risk tier.

In this chapter, we explore "moments"—which include expected values, variances, standard deviations, and covariances—when working with joint, conditional, and marginal discrete distributions. Under the Exam P syllabus, multivariate distributions for joint, conditional, and marginal moments are evaluated in the discrete setting.

1. Joint Expected Values

When working with two discrete random variables, \(X\) and \(Y\), with joint probability mass function \(p(x, y) = P(X = x, Y = y)\), we often want to find the average of a function involving both variables, such as \(E[XY]\) or \(E[X + Y]\). This is a joint expected value.

To find the expected value of any function \(g(X, Y)\), multiply the function's value at each point by the joint probability and sum across all possible values of \(x\) and \(y\):

\( E[g(X, Y)] = \sum_{x} \sum_{y} g(x, y) \cdot p(x, y) \)

Analogy: Imagine a grid of tiles on the floor. Each tile represents a pair \((x, y)\) with a payoff \(g(x, y)\) and a probability weight \(p(x, y)\). To find the overall expected value, you multiply each tile's payoff by its probability weight and sum them all up.

Quick Review: Expectation is always linear: \(E[aX + bY + c] = aE[X] + bE[Y] + c\). This linearity holds whether \(X\) and \(Y\) are independent or dependent!

2. Marginal Moments

Before evaluating conditional relationships or covariance, we frequently need the moments of each variable individually. These are the marginal moments.

First, sum over the other variable to obtain the marginal probability function:

\( p_X(x) = \sum_{y} p(x, y) \quad \text{and} \quad p_Y(y) = \sum_{x} p(x, y) \)

Marginal Expected Value

The marginal mean can be computed directly from the marginal probability function (or directly from the joint sum):

\( E[X] = \sum_{x} x \cdot p_X(x) = \sum_{x} \sum_{y} x \cdot p(x, y) \)

Marginal Variance and Standard Deviation

To compute the marginal variance, compute the second raw moment \(E[X^2] = \sum_x x^2 \cdot p_X(x)\), and then apply the variance formula:

\( Var(X) = E[X^2] - (E[X])^2 \)

The standard deviation is the square root of the variance: \( \sigma_X = \sqrt{Var(X)} \).

3. Conditional Moments

When given specific information about one variable (for instance, that \(Y = y\)), we update our probabilities to the conditional probability function:

\( p_{X|Y}(x|y) = P(X = x \mid Y = y) = \frac{p(x, y)}{p_Y(y)}, \quad \text{for } p_Y(y) > 0 \)

Conditional Expectation

The conditional mean of \(X\) given \(Y = y\) is the expectation taken with respect to this conditional distribution:

\( E[X \mid Y = y] = \sum_{x} x \cdot p_{X|Y}(x|y) \)

Here, \(y\) is treated as a fixed constant while summing over all possible values of \(x\).

Conditional Variance and Standard Deviation

The conditional variance measures the spread of \(X\) given that \(Y = y\):

\( Var(X \mid Y = y) = E[X^2 \mid Y = y] - (E[X \mid Y = y])^2 \)

Where \(E[X^2 \mid Y = y] = \sum_x x^2 \cdot p_{X|Y}(x|y)\). The conditional standard deviation is \( \sigma_{X|Y=y} = \sqrt{Var(X \mid Y = y)} \).

Note: \(E[X \mid Y]\) is itself a random variable because its numerical value depends on the outcome of \(Y\).

4. Covariance and Correlation

Covariance measures the linear relationship between two random variables:

\( Cov(X, Y) = E[(X - E[X])(Y - E[Y])] = E[XY] - E[X]E[Y] \)

Independence: If \(X\) and \(Y\) are independent discrete random variables, then \(p(x, y) = p_X(x)p_Y(y)\), which implies \(E[XY] = E[X]E[Y]\) and therefore \(Cov(X, Y) = 0\). Note that \(Cov(X, Y) = 0\) indicates no linear relationship, but does not necessarily guarantee independence in general.

Correlation Coefficient (\(\rho\))

Correlation standardizes covariance so that it is dimensionless and bounded between -1 and 1:

\( \rho_{XY} = \frac{Cov(X, Y)}{\sigma_X \sigma_Y} = \frac{Cov(X, Y)}{\sqrt{Var(X) \cdot Var(Y)}} \)

5. Double Expectation and Total Variance

These two fundamental identities allow you to evaluate marginal moments by conditioning on an intermediate variable.

The Law of Iterated Expectations

The overall mean is the expected value of the conditional mean:

\( E[X] = E[E[X \mid Y]] = \sum_{y} E[X \mid Y = y] \cdot p_Y(y) \)

The Law of Total Variance

Total variance breaks down into the expectation of the conditional variance plus the variance of the conditional expectation:

\( Var(X) = E[Var(X \mid Y)] + Var(E[X \mid Y]) \)

Memory Aid: Remember this as "EV + VE" (Expected Value of Variance + Variance of Expected Value).

6. Common Mistakes to Avoid

1. Mixing up Marginal and Conditional: Always verify the denominator. The marginal distribution uses the unconditional sum \(p_X(x) = \sum_y p(x, y)\), whereas the conditional distribution requires dividing by the marginal probability of the given condition: \(p_{X|Y}(x|y) = p(x,y) / p_Y(y)\).

2. Forgetting to square the mean: In both marginal variance \(Var(X) = E[X^2] - (E[X])^2\) and conditional variance \(Var(X|Y=y) = E[X^2|Y=y] - (E[X|Y=y])^2\), make sure to subtract the square of the expectation.

3. Assuming \(E[XY] = E[X]E[Y]\): This identity holds whenever \(Cov(X,Y)=0\). Independence is sufficient but not necessary; dependent variables can also be uncorrelated. When variables are dependent, calculate \(E[XY] = \sum_x \sum_y xy \cdot p(x, y)\) using the joint table.

7. Summary Checklist

Before tackling practice problems on multivariate discrete moments, make sure you can:

  • Construct marginal distributions by summing rows or columns of a joint probability table.
  • Calculate marginal expected values, variances, and standard deviations: \(E[X]\), \(Var(X)\), and \(\sigma_X\).
  • Construct conditional probability distributions \(p_{X|Y}(x|y)\) and calculate conditional moments \(E[X \mid Y = y]\) and \(Var(X \mid Y = y)\).
  • Calculate \(E[XY]\), covariance \(Cov(X, Y) = E[XY] - E[X]E[Y]\), and correlation \(\rho_{XY}\).
  • Apply the Law of Iterated Expectations and the Law of Total Variance (EV + VE).