<p>Forecasting is a very interesting topic for economists. There are many techniques, but basically the key is to compare your forecast against a base model, which could be validation data (test data), a naïve model (really basic estimation) or a more complex model.</p>
<p>Forecasters tend to use the Root Mean Squared Error (RMSE) or Mean Absolute Percent Error (MAPE) statistics to compare models, but there are also others like the Akaike Information Criterion (AIC), Bayesian Information Criterio (BIC), etc. You can google them out easily.</p>
<p>Finally, in the case of VAR model, an actual forecast of the data is not the most useful tool, but the Impulse Response Function (IRF) which is basically a simulation of what could happen (response) to a variable if another changes due to a shock (impulse). This response is dynamic (not constant) over time, to the difference of the typical regression coefficients.</p>

CC BY-SA 4.0
2022-05-11T05:33:06.170Z
2022-05-11T05:33:06.170Z
Lima_Institute_of_Econometrics
<p>Let's say I have a time series model (VAR model for example). How can I know that my forecast is good ? I could use the R2 but is there something else? I also know I could just use in sample forecasting and compare my model to the observed data, but is there a problem with this method ?</p>
<p>Thank you</p>

CC BY-SA 4.0
2022-05-09T10:14:47.893Z
How are you defining *quality*? Do you wish to compare forecasts from one model with those with another? In sample forecasting is comparing the $hat y$ with $y$..this is similar to checking $R^2$. In any case, you need to first define quality
Dayne
CC BY-SA 4.0
2022-05-09T13:52:05.467Z
Similar topics are popular on [Cross Validated Stack Exchange](https://stats.stackexchange.com/questions).
Richard Hardy
CC BY-SA 4.0
2023-06-05T12:27:55.070Z
This seems squarely on-topic for Cross Validated, perhaps even a better fit there than here because there is nothing specific to economics in this question; the connection to economics seems to be that economists are among the many people who study time series, VAR models, and forecasting. Given the lack of (upvoted) answers, is a migration warranted? Possible?
Dave
<macroeconomics><econometrics><time-series><forecasting><svar>
How to assess the quality of a forecast?
BAL
# How to assess the quality of a forecast?

(Asked by: username_0 on 2022-05-09)

Let's say I have a time series model (VAR model for example). How can I know that my forecast is good ? I could use the R2 but is there something else? I also know I could just use in sample forecasting and compare my model to the observed data, but is there a problem with this method ?

Thank you

**Tags:** macroeconomics, econometrics, time-series, forecasting, svar
> **Comments:**
> **username_1**: How are you defining *quality*? Do you wish to compare forecasts from one model with those with another? In sample forecasting is comparing the $hat y$ with $y$..this is similar to checking $R^2$. In any case, you need to first define quality  
> **username_2**: Similar topics are popular on [Cross Validated Stack Exchange](https://stats.stackexchange.com/questions).  
> **username_3**: This seems squarely on-topic for Cross Validated, perhaps even a better fit there than here because there is nothing specific to economics in this question; the connection to economics seems to be that economists are among the many people who study time series, VAR models, and forecasting. Given the lack of (upvoted) answers, is a migration warranted? Possible?  

---

## 1 answers

### Answer by username_4 on 2022-05-11. Score: 1

Forecasting is a very interesting topic for economists. There are many techniques, but basically the key is to compare your forecast against a base model, which could be validation data (test data), a naïve model (really basic estimation) or a more complex model.

Forecasters tend to use the Root Mean Squared Error (RMSE) or Mean Absolute Percent Error (MAPE) statistics to compare models, but there are also others like the Akaike Information Criterion (AIC), Bayesian Information Criterio (BIC), etc. You can google them out easily.

Finally, in the case of VAR model, an actual forecast of the data is not the most useful tool, but the Impulse Response Function (IRF) which is basically a simulation of what could happen (response) to a variable if another changes due to a shock (impulse). This response is dynamic (not constant) over time, to the difference of the typical regression coefficients.


---

