Our objective is to develop a methodology to detect regions in risk-return space where the out-of-sample performance of portfolios is consistent with their in-sample performance. We use the Berkowitz statistic to evaluate the accuracy of the density forecast, derived from in-sample portfolio returns, of out-of-sample portfolio returns. Defined by its co-ordinates in risk-return space, a portfolio is ‘consistent ’ if this statistic indicates that the in-sample return density is a good predictor of out-of-sample portfolio returns. Given a track record of portfolio selections by an algorithm, we create a framework that defines a portfolio's position in risk-return space and describes a ‘consistency region ’ bounded by a ‘consistency frontier’. We argue that, for a given level of expected return, a consistent portfolio is preferable to an inconsistent one. This methodology for the creation of a consistency region is independent of the portfolio selection algorithm. We use mean-variance portfolio selection, to firstly validate the approach by using well-behaved simulated multivariate normal returns; variation of the time interval for estimation demonstrates that the consistency region behaves intuitively reasonably. Secondly, we show that with real data, the behaviour of the consistency region is driven by asset return dynamics and consequently less stable. Two investment strategies using consistency are created and compared with a minimum variance portfolio benchmark strategy. In nearly all cases, consistent portfolios lead to out-of-sample outperformance of the benchmark. We also investigate the consistency of the naive diversified (1/N) portfolio.