Evaluating Machine Learning Alternatives To Black-Scholes-Merton For Option Pricing: A Focus On Deep Out-of-the-Money Regimes

Authors: Anmol Bajpai

Abstract: The Black-Scholes-Merton (bsm) model has been the cornerstone of options pricing theory since Black and Scholes (1973) and Merton (1973), yet its assumption of con-stant volatility under Geometric Brownian Motion produces systematic mispricing in the distributional tails — most acutely for deep out-of-the-money (dotm) con-tracts where fat tails and jump risk concentrate. This paper delivers an empirical, quantitative investigation into whether non-parametric machine learning architec-tures can correct this structural deficiency. We construct a synthetic options dataset of 20,000 contracts on a liquid S&P 500–like index, deliberately engineered to replicate the empirically documented volatil-ity smile surface incorporating negative skew, curvature, and term-structure effects. Against this ground truth, we benchmark three models: the standard bsm formula (applied using a flat at-the-money volatility), an XGBoost gradient-boosted tree en-semble, and a two-layer Long Short-Term Memory (LSTM) recurrent neural network. Model selection and hyperparameter optimisation are performed on a time-based split to preclude data leakage. Our central empirical finding is stark: XGBoost reduces dotm Root Mean Squared Error (RmsE) from $99.56 to $17.22 — an 82.7% improvement over bsm — and reduces dotm Mean Absolute Percentage Error (mapE) from 126.27% to 46.82%. The LSTM, conversely, underperforms even the bsm baseline across all moneyness bands, exposing a critical architectural mismatch: sequential models require panel data (repeated observations of the same contract over time), not the cross-sectional option chains typically available in practice. SHAP value decompo-sition of the XGBoost model reveals that BSM price residuals, log-moneyness, and the VIX regime collectively account for the dominant share of pricing decisions in the dotm region, corroborating the practitioner intuition that tail risk is priced through vol-regime conditioning rather than structural parameter updates. We dis-cuss arbitrage constraint violations, computational trade-offs, and the regulatory implications of deploying black-box pricers in risk-management contexts.

DOI: https://zenodo.org/records/23008520