Prediction Intervals for Machine Learning

A prediction interval is a quantification of the uncertainty on a prediction. A prediction from a machine learning perspective is a single point/purpose that hides the uncertainty of that prediction. It provides a probabilistic upper and lower bounds on the estimate of an outcome variable. And are most widely used when making predictions or forecasts with a regression model, where a quantity is being foreseen. It surrounds the prediction created by the model and hopefully covers the range of the actual outcome.

A confidence interval quantifies the unpredictability on an approximated population variable, such as the mean or standard deviation. Whereas a prediction interval evaluates the uncertainty on a single observation approximated from the population.

Prediction intervals provide a way to evaluate and communicate the ambiguity in a prediction. They are different from confidence intervals that instead explore to evaluate the uncertainty in a population parameter such as a mean or standard deviation. Prediction intervals describe the ambiguity for a single specific outcome.

How to Calculate a Prediction Interval

A prediction interval is calculated as some combo of the predicted variance of the model and the variance of the outcome variable.

Prediction intervals are difficult to calculate but, in practice easy to describe.

In linear regression, which is a model that describes the linear combination of inputs to calculate the output variables we can predict the confidence interval directly. In the cases of nonlinear regression algorithms, like artificial neural networks, it is a lot more challenging, requires the choice and implementation of specialized techniques. General techniques such as the bootstrap resampling method can be used, but are computationally expensive to calculate.