Linear Model Measures

The standard linear models for stationary time series are the autoregressive (AR) models and the autoregressive moving average (ARMA) models. Here these models are implemented in order to derive statistical measures of fitting and prediction for varying model orders and prediction steps. The difference between the fitting and prediction measures is in the set of samples that are used to compute the statistical measure: all the samples are used for fitting and the samples in the so-called test set for prediction. Here, the test set is the last part of the time series. For multi-step fit or prediction the iterative scheme is used and the one step predictions from previous steps are used to make predictions for the current step.

Four standard statistical measures are encountered, the mean square error (MSE), the normalized mean square error (NMSE), the normalized root mean square error (NMSE) and the correlation coefficient (CC). The three last measures account for the variance of the time series and this allows comparison of the measure across different time series.   

AR and ARMA models are basic tools of linear analysis of time series and statistical description for them (estimation, inference, predictions) can be found in any standard book on time series analysis, e.g. see

Box G.E.P., Jenkins G.M., and Reinsel G.C. (1994), Time Series Analysis: Forecasting and Control, Prentice-Hall, Third Edition, New Jersey.

 

Autoregressive Fit (AR_____Fit)

Autoregressive Fit gives four statistical measures of goodness of fit with autoregressive (AR) model. A number of different orders for the AR model can be given and the fit can be computed for a number of lead times. The following parameters can be specified:

 - model order (m) : any valid matlab format denoting an array of positive integers or a single positive integer. The default is '1' meaning that the computations of the goodness of fit measures will be done for the simplest model AR(1).

 - prediction time (h) : any valid matlab format denoting an array of positive integers or a single positive integer. The default is '1' meaning that the computations of the goodness of fit measures will be done only for one-step ahead.

The user can activate (check) any of the four statistical measures to save results for. If none is checked then results for all four measures will be computed and the measures will be included in the measure list. The statistical measures are the following:

- MSE : if checked the mean square error (MSE) measure of goodness of fit for the specified model orders and prediction times will be included in the list of measures. MSE is the sum of squares of prediction errors defined as

where xi+h is the actual sample and xi(h) is the h-step ahead prediction at current time i, for i=m,...,N-h, and N is the length of the time series.  

- NMSE : if checked the normalized mean square error (MSE) measure of goodness of fit for the specified model orders and prediction times will be included in the list of measures. NMSE is the MSE divided by the variance of the samples included in the sum of MSE, defined as

where is the mean of the samples in the sum.

- NRMSE : if checked the normalized root mean square error (MSE) measure of goodness of fit for the specified model orders and prediction times will be included in the list of measures. NRMSE is the square root of NMSE.

- CC : if checked the correlation coefficient (CC) measure of goodness of fit for the specified model orders and prediction times will be included in the list of measures. CC is the standard Pearson correlation coefficient between the actual samples and the respective predictions, i.e. xi+h and xi(h) for h-step ahead prediction at current time i.

Example: If the user selects this measure by activating the check box in the beginning of the measure line and sets for model order (m) '5 10' and for prediction time (h) '1:4', and checks only NRMSE then the NRMSE measure of Autoregressive Fit is computed for the combinations of the 2 values of m and the 4 values of h and in the measure list the following measure names will appear

AR_____FitNRMSEm5h1
AR_____FitNRMSEm5h2
AR_____FitNRMSEm5h3
AR_____FitNRMSEm5h4
AR_____FitNRMSEm10h1
AR_____FitNRMSEm10h2
AR_____FitNRMSEm10h3
AR_____FitNRMSEm10h4

 

Autoregressive Prediction (AR_Predict)

Autoregressive Prediction gives four statistical measures for the prediction error with autoregressive (AR) model. The AR model is estimated in the first part of the time series, the so-called training set, and the predictions are made in the second part of the time series, the so-called test set. A parameter can be specified to determine the size of the test set. A number of different orders for the AR model can be given and the predictions can be computed for a number of lead times. The following parameters can be specified:

 - fraction for test set (f) : any number between 0.1 and 0.9. The default is '0.5' meaning that the test set is the second half of the time series. Typically f should be smaller or equal to 0.5, so that the test set does not exceed in size the training set. 

- model order (m) : any valid matlab format denoting an array of positive integers or a single positive integer. The default is '1' meaning that the computations of the goodness of fit (prediction here) measures will be done for the simplest model AR(1).

 - prediction time (h) : any valid matlab format denoting an array of positive integers or a single positive integer. The default is '1' meaning that the computations of the goodness of fit (prediction here) measures will be done only for one-step ahead.

The user can activate (check) any of the four statistical measures to save results for. If none is checked then results for all four measures will be saved and the measures will be included in the measure list. The statistical measures are the following:

- MSE : if checked the mean square error (MSE) measure of prediction in the test set for the specified model orders and prediction times will be included in the list of measures. MSE is the sum of squares of prediction errors defined as

where xi+h is the actual sample and xi(h) is the h-step ahead prediction at current time i, for i=N1,...,N-h, N is the length of the time series and N1 is the length of the training set ( N1 = (1-f)N ). 

- NMSE : if checked the normalized mean square error (MSE) measure of prediction in the test set for the specified model orders and prediction times will be included in the list of measures. NMSE is the MSE divided by the variance of the samples included in the sum of MSE, defined as

where is the mean of the samples in the sum

- NRMSE : if checked the normalized root mean square error (MSE) measure of prediction in the test set for the specified model orders and prediction times will be included in the list of measures. NRMSE is the square root of NMSE.

- CC : if checked the correlation coefficient (CC) measure of prediction in the test set for the specified model orders and prediction times will be included in the list of measures. CC is the standard Pearson correlation coefficient between the actual samples and the respective predictions.

Example: If the user selects this measure by activating the check box in the beginning of the measure line and sets for fraction for test set (f) '0.25', model order (m) '5 10' and for prediction time (h) '1:4', and checks only NRMSE then the NRMSE measure of Autoregressive Prediction is computed on a test set of length one forth of time series length for the combinations of the 2 values of m and the 4 values of h and in the measure list the following measure names will appear (note that the value for f in the measure names is multiplied by 100, and rounded) 

AR_PredictNRMSEf25m5h1
AR_PredictNRMSEf25m5h2
AR_PredictNRMSEf25m5h3
AR_PredictNRMSEf25m5h4
AR_PredictNRMSEf25m10h1
AR_PredictNRMSEf25m10h2
AR_PredictNRMSEf25m10h3
AR_PredictNRMSEf25m10h4

 

Autoregressive Moving Average Fit (ARMA___Fit)

Autoregressive Moving Average Fit gives four statistical measures of goodness of fit with autoregressive moving average (ARMA) model. A number of different orders for the AR part and the MA part of the model can be given and the fit can be computed for a number of lead times. The following parameters can be specified:

 - AR order (m) : any valid matlab format denoting an array of positive integers or a single positive integer. The default is '1' meaning that the computations of the goodness of fit measures will be done for the ARMA model with AR part of order 1.

 - MA order (p) : any valid matlab format denoting an array of positive integers or a single positive integer. The default is '1' meaning that the computations of the goodness of fit measures will be done for the ARMA model with MA part of order 1.

 - prediction time (h) : any valid matlab format denoting an array of positive integers or a single positive integer. The default is '1' meaning that the computations of the goodness of fit measures will be done only for one-step ahead.

The user can activate (check) any of the four statistical measures to save results for. If none is checked then results for all four measures will be computed and the measures will be included in the measure list. The statistical measures are the following:

- MSE : if checked the mean square error (MSE) measure of goodness of fit for the specified model orders and prediction times will be included in the list of measures. MSE is the sum of squares of prediction errors defined as

where xi+h is the actual sample and xi(h) is the h-step ahead prediction at current time i, for i=m,...,N-h, and N is the length of the time series.  

- NMSE : if checked the normalized mean square error (MSE) measure of goodness of fit for the specified model orders and prediction times will be included in the list of measures. NMSE is the MSE divided by the variance of the samples included in the sum of MSE, defined as

where is the mean of the samples in the sum.

- NRMSE : if checked the normalized root mean square error (MSE) measure of goodness of fit for the specified model orders and prediction times will be included in the list of measures. NRMSE is the square root of NMSE.

- CC : if checked the correlation coefficient (CC) measure of goodness of fit for the specified model orders and prediction times will be included in the list of measures. CC is the standard Pearson correlation coefficient between the actual samples and the respective predictions, i.e. xi+h and xi(h) for h-step ahead prediction at current time i.

Example: If the user selects this measure by activating the check box in the beginning of the measure line and sets for order of the AR part (m) '5 10', the order for the MA part (p) '1' and for prediction time (h) '1:4', and checks only NRMSE then the NRMSE measure of Autoregressive Moving Average Fit is computed for the combinations of the 2 values of m, and the 4 values of h (p has only one value here) and in the measure list the following measure names will appear

ARMA___FitNRMSEm5p1h1
ARMA___FitNRMSEm5p1h2
ARMA___FitNRMSEm5p1h3
ARMA___FitNRMSEm5p1h4
ARMA___FitNRMSEm10p1h1
ARMA___FitNRMSEm10p1h2
ARMA___FitNRMSEm10p1h3
ARMA___FitNRMSEm10p1h4

 

Autoregressive Moving Average Prediction (ARMAPredic)

Autoregressive Moving Average Prediction gives four statistical measures for the prediction error with autoregressive moving average (ARMA) model. The ARMA model is estimated in the first part of the time series, the so-called training set, and the predictions are made in the second part of the time series, the so-called test set. A parameter can be specified to determine the size of the test set. A number of different orders for the AR part and the MA part of the model can be given and the predictions can be computed for a number of lead times. The following parameters can be specified:

 - fraction for test set (f) : any number between 0.1 and 0.9. The default is '0.5' meaning that the test set is the second half of the time series. Typically f should be smaller or equal to 0.5, so that the test set does not exceed in size the training set. 

- AR order (m) : any valid matlab format denoting an array of positive integers or a single positive integer. The default is '1' meaning that the computations of the goodness of fit (prediction here) measures will be done for the ARMA model with AR part of order 1.

 - MA order (p) : any valid matlab format denoting an array of positive integers or a single positive integer. The default is '1' meaning that the computations of the goodness of fit (prediction here) measures will be done for the ARMA model with MA part of order 1.

 - prediction time (h) : any valid matlab format denoting an array of positive integers or a single positive integer. The default is '1' meaning that the computations of the goodness of fit (prediction here) measures will be done only for one-step ahead.

The user can activate (check) any of the four statistical measures to save results for. If none is checked then results for all four measures will be saved and the measures will be included in the measure list. The statistical measures are the following:

- MSE : if checked the mean square error (MSE) measure of prediction in the test set for the specified model orders and prediction times will be included in the list of measures. MSE is the sum of squares of prediction errors defined as

where xi+h is the actual sample and xi(h) is the h-step ahead prediction at current time i, for i=N1,...,N-h, N is the length of the time series and N1 is the length of the training set ( N1 = (1-f)N ). 

- NMSE : if checked the normalized mean square error (MSE) measure of prediction in the test set for the specified model orders and prediction times will be included in the list of measures. NMSE is the MSE divided by the variance of the samples included in the sum of MSE, defined as

where is the mean of the samples in the sum

- NRMSE : if checked the normalized root mean square error (MSE) measure of prediction in the test set for the specified model orders and prediction times will be included in the list of measures. NRMSE is the square root of NMSE.

- CC : if checked the correlation coefficient (CC) measure of prediction in the test set for the specified model orders and prediction times will be included in the list of measures. CC is the standard Pearson correlation coefficient between the actual samples and the respective predictions.

Example: If the user selects this measure by activating the check box in the beginning of the measure line and sets for fraction for test set (f) '0.25', for the order of the AR part (m) '5 10', for the order for the MA part (p) '1', and for prediction time (h) '1:4', and checks only NRMSE then the NRMSE measure of Autoregressive Moving Average Prediction is computed on a test set of length one forth of time series length for the combinations of the 2 values of m and the 4 values of h (p has only one value here) and in the measure list the following measure names will appear (note that the value for f in the measure names is multiplied by 100, and rounded) 

ARMAPredicNRMSEf25m5p1h1
ARMAPredicNRMSEf25m5p1h2
ARMAPredicNRMSEf25m5p1h3
ARMAPredicNRMSEf25m5p1h4
ARMAPredicNRMSEf25m10p1h1
ARMAPredicNRMSEf25m10p1h2
ARMAPredicNRMSEf25m10p1h3
ARMAPredicNRMSEf25m10p1h4

 

OK

By pressing this button the window of "Linear Model Measures" will disappear and the user will be moved to the "Select / run measures" window. Any changes in the measures and parameter values will be stored.
 

Cancel

Quit without doing anything and return to the "Select / run measures" window. Any changes in the measures and parameter values will be ignored.
 

Help

This file will be shown.