Effect Sizes

Effect sizes quantify the magnitude of differences in standardized units, making comparisons meaningful across different scales.

Cohen's d

Feature Name: cohens_d

\[ d = \frac{\mu_{\text{post}} - \mu_{\text{pre}}}{s_{\text{pooled}}} \]

where the pooled standard deviation is: $$ s_{\text{pooled}} = \sqrt{\frac{(n_1-1)s_1^2 + (n_2-1)s_2^2}{n_1 + n_2 - 2}} $$

Interpretation (Cohen's Conventions)

Value	Interpretation
|d| ≈ 0.2	Small effect
|d| ≈ 0.5	Medium effect
|d| ≈ 0.8	Large effect
|d| > 1.0	Very large effect

Properties

• Range: (-∞, +∞)
• Independent of sample size (unlike p-values)

Why Useful

Cohen's d provides a standardized measure of how "different" the post-break segment is from the pre-break segment. It won't show artificial significance with large samples.

Used by: All 25 detectors

---

Glass's Delta

Feature Name: glass_delta

\[ \Delta = \frac{\mu_{\text{post}} - \mu_{\text{pre}}}{s_{\text{pre}}} \]

Key Difference from Cohen's d

Uses only the pre-break standard deviation as denominator. Appropriate when:

1. The break is expected to change variance
2. Pre-break represents "normal" regime

Why Useful

Captures how many pre-break standard deviations the mean has shifted — relevant when the "before" period represents baseline behavior.

Used by: mlp_ensemble_deep_features, meta_stacking_7models

---

Hedges' g

Feature Name: hedges_g

\[ g = d \times \left(1 - \frac{3}{4(n_1 + n_2) - 9}\right) \]

Why Use Hedges' g?

• Bias-corrected version of Cohen's d
• Cohen's d slightly overestimates effect in small samples
• More accurate for small samples (n < 20)

Used by: meta_stacking_7models

---

Cliff's Delta

Feature Name: cliffs_delta

\[ \delta = \frac{\#\{x_i > y_j\} - \#\{x_i < y_j\}}{n_1 \times n_2} \]

Interpretation

Value	Interpretation
δ = 0	Groups overlap completely
δ = 1	All values in group 2 exceed group 1
δ = -1	All values in group 1 exceed group 2

Effect Size Thresholds

Range	Interpretation
|δ| < 0.147	Negligible
|δ| < 0.33	Small
|δ| < 0.474	Medium
|δ| ≥ 0.474	Large

Why Useful

Non-parametric effect size — makes no assumptions about distribution shape. Robust to outliers.

Used by: meta_stacking_7models