Publication detail

A Bayesian Approach to Backtest Overfitting

Author(s): RNDr. Jiří Witzany Ph.D.,
Type: IES Working Papers
Year: 2017
Number: 18
ISSN / ISBN:
Published in: IES Working Papers 18/2017
Publishing place: Prague
Keywords: Backtest, multiple testing, bootstrapping, cross-validation, probability of backtest overfitting, investment strategy, optimization, Sharpe ratio, Bayesian probability, MCMC
JEL codes: G1, G2, C5, G24, C11, C12, C52
Suggested Citation: Witzany J. (2017). " A Bayesian Approach to Backtest Overfitting” IES Working Paper 18/2017. IES FSV. Charles University.
Abstract: Quantitative investment strategies are often selected from a broad class of candidate models estimated and tested on historical data. Standard statistical technique to prevent model overfitting such as out-sample back-testing turns out to be unreliable in the situation when selection is based on results of too many models tested on the holdout sample. There is an ongoing discussion how to estimate the probability of back-test overfitting and adjust the expected performance indicators like Sharpe ratio in order to reflect properly the effect of multiple testing. We propose a consistent Bayesian approach that consistently yields the desired robust estimates based on an MCMC simulation. The approach is tested on a class of technical trading strategies where a seemingly profitable strategy can be selected in the naïve approach.
Downloadable: wp_2017_18_witzany
December 2020
MonTueWedThuFriSatSun
 123456
78910111213
14151617181920
21222324252627
28293031   

Partners

Deloitte

Sponsors

CRIF
McKinsey
Patria Finance