Welcome to Linear Combinations of Random Variables!
Hello! Today we are going to explore how to combine different random variables. Think of this chapter as "statistical Lego." We are taking individual blocks of data (like the height of one student) and learning what happens when we stack them together or scale them up (like the total height of a classroom).
Don't worry if this seems a bit abstract at first. By the end of these notes, you'll see that it’s mostly just following a few golden rules. Let’s dive in!
1. Scaling and Shifting a Single Variable
Imagine you have a random variable \(X\). This could be the weight of an apple. What happens if we double the weight of every apple (\(2X\)) or add a 10g sticker to every apple (\(X + 10\))?
The Mean (Expectation)
The Expectation, or \(E(X)\), is very "well-behaved." It does exactly what you expect it to do. If you multiply the data by \(a\) and add \(b\), the mean does the same.
The Formula: \(E(aX + b) = aE(X) + b\)
The Variance
The Variance, or \(Var(X)\), measures spread. If you add a constant \(b\) to every value, the spread doesn't change (the whole graph just slides over). However, if you multiply by \(a\), the spread changes by \(a^2\).
The Formula: \(Var(aX + b) = a^2Var(X)\)
Example: If \(E(X) = 10\) and \(Var(X) = 4\), what is the mean and variance of \(3X + 5\)?
1. New Mean: \(3(10) + 5 = 35\)
2. New Variance: \(3^2 \times 4 = 9 \times 4 = 36\)
Quick Tip: Always remember to square the multiplier when calculating variance! The constant \(b\) is ignored for variance because shifting data doesn't make it more or less "spread out."
Key Takeaway: Means follow the formula exactly; Variances ignore the addition/subtraction and square the multiplier.
2. Combining Two or More Variables
Now, what if we have two different variables, \(X\) and \(Y\)? For example, \(X\) is the weight of a cereal box and \(Y\) is the weight of the toy inside.
The Expectation of a Sum
Again, the mean is very friendly. To find the total mean, just add the individual means.
\(E(X + Y) = E(X) + E(Y)\)
\(E(X - Y) = E(X) - E(Y)\)
The Variance of a Sum (For Independent Variables)
This is where students often trip up! If \(X\) and \(Y\) are independent (meaning one doesn't affect the other), their variances always add, even if you are subtracting the variables.
The Formula:
\(Var(X + Y) = Var(X) + Var(Y)\)
\(Var(X - Y) = Var(X) + Var(Y)\)
Analogy: The "Uncertainty" Rule
Think of variance as "uncertainty" or "error." If you add two items, you have more uncertainty. If you subtract one item from another, you still have two sources of error, so the total uncertainty still goes up. You can't subtract uncertainty!
Did you know? In the Cambridge 9709 syllabus, you can only use these variance formulas if the variables are independent. Look for that word in the question!
Key Takeaway: Means can be added or subtracted. Variances are always added (provided the variables are independent).
3. The "Double" vs. "Two Different" Trap
This is the most common mistake in the entire chapter. Pay close attention!
Case A: \(2X\) (One item doubled)
This is one apple that we magically make twice as heavy.
\(Var(2X) = 2^2 \times Var(X) = 4Var(X)\)
Case B: \(X_1 + X_2\) (Two different apples)
These are two separate apples from the same tree. They are independent.
\(Var(X_1 + X_2) = Var(X_1) + Var(X_2) = 2Var(X)\)
Notice the difference: \(4Var(X)\) is much larger than \(2Var(X)\). Doubling one variable is riskier and more spread out than adding two separate variables together.
Key Takeaway: Multiplying a variable by \(n\) is not the same as adding \(n\) independent copies of that variable.
4. Linear Combinations of Normal Distributions
If your variables \(X\) and \(Y\) follow a Normal Distribution, then any linear combination of them (like \(2X + 3Y\)) will also follow a Normal Distribution. This is super helpful because it means we can still use our Z-tables!
Step-by-Step Process for Normal Distribution Problems:
1. Find the new Mean: Use the \(E(aX + bY)\) rules.
2. Find the new Variance: Use the \(Var(aX + bY)\) rules.
3. Write the new distribution: \(W \sim N(\mu_{new}, \sigma^2_{new})\).
4. Standardize: Use \(Z = \frac{W - \mu}{\sigma}\) (remember to use the standard deviation, which is the square root of your new variance).
5. Find the probability: Look up the value in your Normal Distribution table.
Example:
Let \(X \sim N(10, 4)\) and \(Y \sim N(12, 9)\). Find \(P(X + Y > 25)\).
- New Mean: \(10 + 12 = 22\)
- New Variance: \(4 + 9 = 13\)
- New Distribution: \(S \sim N(22, 13)\)
- Standardize: \(Z = \frac{25 - 22}{\sqrt{13}} = \frac{3}{3.606} = 0.832\)
- Now solve for \(P(Z > 0.832)\) as usual.
Key Takeaway: If the inputs are Normal, the output is Normal. Just calculate the new mean and new variance first!
5. Sum of Independent Poisson Variables
If you have two independent Poisson variables, \(X \sim Po(\lambda_1)\) and \(Y \sim Po(\lambda_2)\), their sum is also Poisson!
The Formula: \(X + Y \sim Po(\lambda_1 + \lambda_2)\)
Note: This only works for adding variables, not for subtraction or multiplying by a constant.
Common Mistakes to Avoid
• Forgetting to square \(a\): In variance, \(Var(3X)\) is \(9Var(X)\), not \(3Var(X)\).
• Subtracting Variances: Even if the question asks for \(Var(X - Y)\), you add the variances.
• Standard Deviation vs Variance: The Normal Distribution notation is \(N(\mu, \sigma^2)\). If the question says the standard deviation is 5, your variance is 25! Always check which one you have before starting your calculations.
• Confusing \(nX\) with \(X_1 + X_2 + ... + X_n\): Read the question carefully. Are you multiplying one measurement by \(n\), or are you adding \(n\) separate measurements?
Quick Review Box
Expectation: \(E(aX + bY + c) = aE(X) + bE(Y) + c\)
Variance: \(Var(aX \pm bY \pm c) = a^2Var(X) + b^2Var(Y)\) (only if independent)
Normal: Combinations of Normal variables are always Normal.
Poisson: The sum of Poisson variables is Poisson (add the \(\lambda\) values).
You've got this! Practice a few problems using these steps, and you'll find that these "linear combinations" are just a logical set of rules to follow. Good luck!