In time series forecasting, the naive forecast - where the forecast for all future periods is set equal to the value from the current period - is the most simple of all forecasting methods. It generally gets a bad wrap, due in large part to this simplicity - how can such a basic methodology be any good? In many applications, its reputation is often justified by - at best - average performance, especially when forecasting many periods ahead. Forecasting inventories (such as reservoir storage levels) is one obvious example. Forecasting inventory levels in 6 months to be equal to todays leaves much information on the table that could easily be used to get a better, more accurate forecast.
However, in a number of applications the naive forecast performs surprisingly well. One example of this are market prices, of which power market prices are one example. Indeed, more complex forecasting methods often struggle to do better (and sometimes do even worse) than the naive in these cases.
Why might this be? A (partial) answer lies in the markets themselves. In well functioning markets, the market price at any time represents the summation of all information available to the market. It can be thought of as an aggregate market “view” of what the price is, given such “underlying” information. For the next period, assuming relatively moderate changes in the underlying information, we may reasonably expect relatively moderate changes in market prices. In such a case, the naive forecast can be a fairly good short term predictor.
A Simple Example
To illustrate this, lets look at the day-ahead market power price in the Nordic markets. To make things a little simpler, we drop the weekend data, and build a simple naive model to predict the next-weekday’s system spot price:
forecast(t+1) = observed(t)
Thus, for example, the forecast for Thursday is Wednesday’s price, and for Monday is the previous Friday’s price. The predictions for our test dataset (May-August 2021) are shown below.
We can test this against a simple feed forward neural network model, trained on a set of historical price features that exhibit some (auto)correlation with current price. The forecasts from this model for our test data set are given below.
There is very little difference in the forecasts of the two models - indeed, the NN model has essentially learned a naive forecast with occasional small modifications. We can see this in more detail examining a set of forecast metrics for the two models:
This is not an atypical result in time-series forecasting using standardised machine learning - such models that are based on recent historical data and historical data patterns end up learning essentially a simple ARIMA-type model. The naive forecast is of course essentially already a simple autoregressive model, and thus performs fairly similarly to these simple ML alternatives.
This highlights a very common issue with many ML time-series forecasting models (in power markets and elsewhere) that appear fairly accurate, but are in reality offering little more (or even no more) than the naive forecast. And it raises the obvious next question is: how do we beat the naive forecast? We will tackle this issue in our next post.
Comments