Week 45 Day 3: Historical Backtesting vs. Monte Carlo: Two Ways to Stress-Test

Historical backtest: 'How would my plan have performed if I retired in January 1966 (followed by a terrible 15-year market)?' Monte Carlo: 'How does my plan perform across 10,000 randomly generated market scenarios, including scenarios that never actually happened?' The backtest is limited to the approximately 100 years of recorded history. The Monte Carlo can imagine scenarios worse (or better) than anything that has ever occurred.

Historical backtesting strengths and weaknesses: Strengths: (a) Uses REAL market data with actual volatility, correlations, and fat tails (no distributional assumptions). (b) Includes the worst historical periods (1929, 1966, 2000, 2008). (c) Intuitive: 'Would I have survived retiring in 1929?' Weaknesses: (a) Only approximately 100 years of U.S. data = only approximately 3-4 independent 30-year retirement periods. Sample size is tiny. (b) The worst historical period may not be the worst FUTURE period. (c) Biased by the U.S. experience (best equity market in the world -- survivorship bias). Monte Carlo strengths and weaknesses: Strengths: (a) Unlimited number of scenarios. Tests thousands of possibilities, including unprecedented ones. (b) Can incorporate forward-looking return assumptions. (c) Easy to adjust inputs (what if returns are 2% lower? what if inflation is higher?). Weaknesses: (a) Depends entirely on input assumptions. Garbage in, garbage out. (b) Standard versions assume normal distributions and independent returns (both unrealistic). (c) May not capture rare extreme events (fat tails) unless explicitly modeled. Best practice: Run BOTH. (1) Historical backtest: confirm your plan survives all known worst cases (cFIREsim, FIRECalc). (2) Monte Carlo with conservative inputs: confirm your plan has >= 85% success across a wide range of hypothetical scenarios (Portfolio Visualizer, Boldin). If your plan passes both tests, it is robust. If it fails either, adjust until it passes both.

The debate between historical backtesting and Monte Carlo simulation reflects deeper epistemological questions about how to model an unknowable future. Historical backtesting treats the past as the set of all possible scenarios ('ergodic assumption'): if your plan survives every historical 30-year period, it will survive the future. This assumption is problematic because: (a) the sample size is small (approximately 4 non-overlapping 30-year periods since 1926), (b) future structural conditions may differ from the past (different monetary systems, demographics, technology, globalization), and (c) the U.S. data set is survivor-biased (Dimson, Marsh, and Staunton, 2002). Monte Carlo treats the future as a draw from a parameterized probability distribution. This approach allows unlimited sample sizes and flexible input assumptions but depends critically on the assumed distribution (normal vs. fat-tailed, independent vs. autocorrelated) and the return/volatility estimates (historical vs. forward-looking). Blanchett and Blanchett (2008) proposed a hybrid approach: bootstrap Monte Carlo (randomly sampling from historical return data rather than from an assumed distribution). This preserves the empirical return distribution (including fat tails and non-normality) while generating thousands of scenarios (overcoming the historical sample size limitation). Block bootstrapping (sampling consecutive multi-year blocks) additionally preserves serial correlation and regime structure. This hybrid approach is arguably the most robust simulation methodology for retirement planning, combining the realism of historical data with the statistical power of Monte Carlo sampling.