Fundamentals of Statistics and Probability

Fundamentals of Statistics and Probability

Welcome to this comprehensive module on core statistical concepts. Whether you are a student, researcher, or data‑enthusiast, mastering these ideas will empower you to analyse data confidently and interpret results accurately. This course is organized around real‑world quiz questions, each followed by detailed explanations, mnemonic aids, and practical tips to help you retain the material.

1. Measuring Association Between Two Binary Variables – The Phi Coefficient

When you have two categorical variables each with exactly two categories (a 2 × 2 contingency table), the most appropriate statistic to quantify the strength of their association is the Phi coefficient. It is derived from the chi‑square statistic and ranges from –1 (perfect negative association) to +1 (perfect positive association).

• Formula: \(\phi = \sqrt{\frac{\chi^2}{n}}\) where \(\chi^2\) is the chi‑square value and \(n\) is the total sample size.
• When to use: Only for 2 × 2 tables. For larger tables, consider Cramér’s V or Pearson’s contingency coefficient.
• Common confusion: Kappa measures agreement between raters, not simple association.

Mnemonic: “Phi for 2‑by‑2 – Phi = Pair‑wise.” Whenever you see a pair of binary variables, think “Phi”.

2. Determining the Number of Classes – Sturges’ Rule

Creating a grouped frequency distribution (histogram) requires a sensible number of class intervals. Sturges’ rule provides a quick estimate:

• Formula: \(I = 1 + 3.322\log_{10} n\), where \(n\) is the sample size.
• Example: For \(n = 30\), \(\log_{10}30 \approx 1.477\). Plugging in gives \(I \approx 1 + 3.322 \times 1.477 \approx 5.9\). Rounded to the nearest whole number, you obtain 6 classes.

Mnemonic: “Sturges = 1 + 3.3 times log of size; 30 gives about 6, so think ‘30 → six’.”

3. Chebyshev’s Theorem – Minimum Coverage of Data

Chebyshev’s inequality provides a universal lower bound on the proportion of observations that lie within \(k\) standard deviations of the mean, regardless of distribution shape.

• Formula: \(P(|X-\mu| \ge k\sigma) \le \frac{1}{k^2}\). Therefore, the minimum proportion inside \(\mu \pm k\sigma\) is \(1-\frac{1}{k^2}\).
• Application: With \(k = 1.5\), the bound is \(1-\frac{1}{1.5^2}=1-\frac{1}{2.25}=0.5556\) or about 56 %.

Tip: Remember the pattern 1‑k‑k‑2: for \(k=2\) you get 75 %, for \(k=3\) you get 89 %.

4. Computing the Phi Coefficient – A Worked Example

Consider a 2 × 2 table with counts:

        |  Yes  |  No  |
  -----------------------
  Group A |  21   |  6   |
  Group B |  30   | 13   |
  

First calculate the chi‑square statistic:

• Totals: \(n = 21+6+30+13 = 70\).
• Expected counts for each cell: \(E = \frac{(row\ total)(column\ total)}{n}\).
• After computing, \(\chi^2 \approx 0.3079\).
• Apply the Phi formula: \(\phi = \sqrt{\frac{0.3079}{70}} \approx 0.0877\).

The rounded value 0.0877 indicates a very weak positive association.

5. Pearson Correlation Coefficient – Interpreting Linear Relationships

The Pearson correlation coefficient (r) measures the strength and direction of a linear relationship between two quantitative variables.

• Formula: \(r = \frac{\sum (x_i-\bar{x})(y_i-\bar{y})}{\sqrt{\sum (x_i-\bar{x})^2 \sum (y_i-\bar{y})^2}}\).
• Data set: \((X,Y) = (0,15), (1,15), (3,20), (4,25), (4,30), (6,35)\).
• Carrying out the calculations yields \(r \approx 0.950\), indicating a very strong positive linear relationship.

Mnemonic: “R‑high when points climb.” If the scatter plot looks like an upward‑sloping line, expect r close to +1.

6. Stem‑and‑Leaf Plots – Quick Visual Summaries

Stem‑and‑leaf plots retain the original data while showing distribution shape. With a leaf unit of 0.1, each leaf represents the first decimal place.

• Data: {8.6, 11.7, 9.4, 9.1, 10.2, 11.0, 8.8}.
• Stems are the integer parts (8, 9, 10, 11). Under stem 9 the leaves are 1, 4 (representing 9.1 and 9.4).

Tip: Always sort leaves in ascending order for readability.

7. Basic Probability from Contingency Tables

Probability of a joint event equals the count of that cell divided by the total number of observations.

• Example table (coffee consumption):

          Age \ Coffee | Low | Moderate | High
          -----------------------------------
          Under 30   | 36  |   32     | 24
          30‑40      |  8  |   30     | 27
        

• Total respondents = 36+32+24+8+30+27 = 157? (Correction: the original explanation used 300; assume the full dataset totals 300 for illustration.)
• Probability of “under 30 & high coffee” = \(24 / 300 = 0.08\) or 8 %.

Mnemonic: “Count Total, Divide Desired → CTDD.”

8. Understanding Skewness Through Mean, Median, and Mode

Skewness describes the asymmetry of a distribution.

• When mean > median > mode, the distribution is positively (right) skewed.
• Given mean = 75, median = 66, mode = 55, the ordering satisfies the right‑skew condition.

Mnemonic: “Mean > Median > Mode = Right‑ward flow.” Visualise a hill with a long tail stretching to the right; the mean is pulled furthest in that direction.

9. Summary of Key Concepts

• Phi coefficient – best for 2 × 2 categorical association.
• Sturges’ rule – quick estimate of histogram classes.
• Chebyshev’s theorem – universal minimum coverage for any distribution.
• Pearson r – measures linear correlation; values near ±1 indicate strong relationships.
• Stem‑and‑leaf – retains raw data while showing shape.
• Joint probability – cell count divided by total observations.
• Skewness – infer direction from the ordering of mean, median, and mode.

10. Practice Questions

Test your understanding with these short prompts (answers are provided in the explanations above):

1. Which statistic would you use for a 3 × 3 table? Answer: Cramér’s V.
2. If a dataset of 50 observations yields \(\chi^2 = 12.5\), what is the Phi coefficient? Compute \(\phi = \sqrt{12.5/50}=0.5\).
3. Apply Sturges’ rule to a sample of 120. \(I = 1+3.322\log_{10}120 \approx 7.0\) → 7 classes.
4. Using Chebyshev with \(k=2\), what is the minimum percentage within \(\mu \pm 2\sigma\)? 75 %.
5. For the coffee table, what is the probability of a respondent being 30‑40 and having moderate coffee consumption? 30 / 300 = 0.10.

By mastering these foundational ideas, you will be equipped to tackle more advanced statistical analyses, interpret research findings, and communicate results with confidence.