Using the Neural Network Time Series Tool

Published on January 2017 | Categories: Documents | Downloads: 34 | Comments: 0 | Views: 267
of 25
Download PDF   Embed   Report

Comments

Content

Using the Neural Network Time Series Tool
1.
If needed, open the Neural Network Start GUI with this command:
2.nnstart

3.

Click Time Series Tool to open the Neural Network Time Series Tool.
(You can also use the commandntstool.)

Notice that this opening pane is different than the opening panes for the
other GUIs. This is because ntstoolcan be used to solve three
different kinds of time series problems.

In the first type of time series problem, you would like to predict
future values of a time series y(t) from past values of that time series
and past values of a second time series x(t). This form of prediction is
called nonlinear autoregressive with exogenous (external) input, or
NARX (see "NARX Network" (narxnet, closeloop)), and can be written as
follows:
y(t) = f(y(t – 1), ..., y(t – d), x(t – 1), ..., (t – d))
This model could be used to predict future values of a stock or bond,
based on such economic variables as unemployment rates, GDP, etc. It
could also be used for system identification, in which models are
developed to represent dynamic systems, such as chemical processes,
manufacturing systems, robotics, aerospace vehicles, etc.

In the second type of time series problem, there is only one
series involved. The future values of a time series y(t) are predicted only
from past values of that series. This form of prediction is called nonlinear
autoregressive, or NAR, and can be written as follows:
y(t) = f(y(t – 1), ..., y(t – d))

This model could also be used to predict financial instruments, but
without the use of a companion series.

The third time series problem is similar to the first type, in that
two series are involved, an input series x(t) and an output/target
series y(t). Here you want to predict values of y(t) from previous values
of x(t), but without knowledge of previous values of y(t). This input/output
model can be written as follows:
y(t) = f(x(t – 1), ..., x(t – d))
The NARX model will provide better predictions than this input-output
model, because it uses the additional information contained in the
previous values of y(t). However, there may be some applications in
which the previous values of y(t) would not be available. Those are the
only cases where you would want to use the input-output model instead
of the NARX model.
4.
For this example, select the NARX model and click Next to proceed.

5.

Click Load Example Data Set in the Select Data window. The Time
Series Data Set Chooser window opens.
Note Use the Inputs and Targets options in the Select Data window when
you need to load data from the MATLAB® workspace.

6.

7.

Select pH Neutralization Process, and click Import. This returns
you to the Select Data window.
8.
Click Next to open the Validation and Test Data window, shown in the
following figure.
The validation and test data sets are each set to 15% of the original
data.

With these settings, the input vectors and target vectors will be randomly
divided into three sets as follows:

70% will be used for training.


15% will be used to validate that the network is generalizing
and to stop training before overfitting.

The last 15% will be used as a completely independent test of
network generalization.
(See "Dividing the Data" for more discussion of the data division
process.)
9.
Click Next.

The standard NARX network is a two-layer feedforward network, with a
sigmoid transfer function in the hidden layer and a linear transfer
function in the output layer. This network also uses tapped delay lines to
store previous values of the x(t) and y(t) sequences. Note that the output
of the NARX network, y(t), is fed back to the input of the network
(through delays), since y(t) is a function of y(t – 1), y(t – 2), ..., y(t – d).
However, for efficient training this feedback loop can be opened.
Because the true output is available during the training of the network,
you can use the open-loop architecture shown above, in which the true
output is used instead of feeding back the estimated output. This has
two advantages. The first is that the input to the feedforward network is
more accurate. The second is that the resulting network has a purely
feedforward architecture, and therefore a more efficient algorithm can be
used for training. This network is discussed in more detail in "NARX
Network" (narxnet, closeloop).
The default number of hidden neurons is set to 10. The default number
of delays is 2. Change this value to 4. You might want to adjust these
numbers if the network training performance is poor.
10. Click Next.

11. Select a training algorithm, then click Train.. Levenberg-Marquardt
(trainlm) is recommended for most problems, but for some noisy and
small problems Bayesian Regularization (trainbr) can take longer but
obtain a better solution. For large problems, however, Scaled Conjugate
Gradient (trainscg) is recommended as it uses gradient calculations
which are more memory efficient than the Jacobian calculations the
other two algorithms use. This example uses the default LevenbergMarquardt.
The training continued until the validation error failed to decrease for six
iterations (validation stop).

12. Under Plots, click Error Autocorrelation. This is used to validate the
network performance.
The following plot displays the error autocorrelation function. It describes
how the prediction errors are related in time. For a perfect prediction

model, there should only be one nonzero value of the autocorrelation
function, and it should occur at zero lag. (This is the mean square error.)
This would mean that the prediction errors were completely uncorrelated
with each other (white noise). If there was significant correlation in the
prediction errors, then it should be possible to improve the prediction perhaps by increasing the number of delays in the tapped delay lines. In
this case, the correlations, except for the one at zero lag, fall
approximately within the 95% confidence limits around zero, so the
model seems to be adequate. If even more accurate results were
required, you could retrain the network by clicking Retrain in ntstool.
This will change the initial weights and biases of the network, and may
produce an improved network after retraining.

13. View the input-error cross-correlation function to obtain additional
verification of network performance. Under the Plots pane, click InputError Cross-correlation.

This input-error cross-correlation function illustrates how the errors are
correlated with the input sequence x(t). For a perfect prediction model,
all of the correlations should be zero. If the input is correlated with the
error, then it should be possible to improve the prediction, perhaps by
increasing the number of delays in the tapped delay lines. In this case,
all of the correlations fall within the confidence bounds around zero.
14. Under Plots, click Time Series Response. This displays the inputs,
targets and errors versus time. It also indicates which time points were
selected for training, testing and validation.

15. Click Next in the Neural Network Time Series Tool to evaluate the
network.

At this point, you can test the network against new data.
If you are dissatisfied with the network's performance on the original or
new data, you can do any of the following:

Train it again.



Increase the number of neurons and/or the number of delays.

Get a larger training data set.
If the performance on the training set is good, but the test set
performance is significantly worse, which could indicate overfitting, then
reducing the number of neurons can improve your results.
16. If you are satisfied with the network performance, click Next.
17. Use this panel to generate a MATLAB function or Simulink ® diagram
for simulating your neural network. You can use the generated code or
diagram to better understand how your neural network computes outputs
from inputs, or deploy the network with MATLAB Compiler™ tools and
other MATLAB and Simulink code generation tools.

18. Use the buttons on this screen to generate scripts or to save your
results.



You can click Simple Script or Advanced Script to create
MATLAB code that can be used to reproduce all of the previous steps
from the command line. Creating MATLAB code can be helpful if you
want to learn how to use the command-line functionality of the toolbox to
customize the training process. In Using Command-Line Functions, you
will investigate the generated scripts in more detail.

You can also have the network saved as net in the workspace.
You can perform additional tests on it or put it to work on new inputs.
19. After creating MATLAB code and saving your results, click Finish.
Using Command-Line Functions
The easiest way to learn how to use the command-line functionality of the
toolbox is to generate scripts from the GUIs, and then modify them to
customize the network training. As an example, look at the simple script
that was created at step 15 of the previous section.
% Solve an Autoregression Problem with External
% Input with a NARX Neural Network

% Script generated by NTSTOOL
%
% This script assumes the variables on the right of
% these equalities are defined:
%
%
phInputs - input time series.
%
phTargets - feedback time series.
inputSeries = phInputs;
targetSeries = phTargets;
% Create a Nonlinear Autoregressive Network with
External Input
inputDelays = 1:4;
feedbackDelays = 1:4;
hiddenLayerSize = 10;
net =
narxnet(inputDelays,feedbackDelays,hiddenLayerSize);
% Prepare the Data for Training and Simulation
% The function PREPARETS prepares time series data
% for a particular network, shifting time by the
minimum
% amount to fill input states and layer states.
% Using PREPARETS allows you to keep your original
% time series data unchanged, while easily customizing
it
% for networks with differing numbers of delays, with
% open loop or closed loop feedback modes.
[inputs,inputStates,layerStates,targets] = ...
preparets(net,inputSeries,{},targetSeries);
% Set up Division of Data for Training, Validation,
Testing
net.divideParam.trainRatio = 70/100;
net.divideParam.valRatio = 15/100;
net.divideParam.testRatio = 15/100;
% Train the Network

[net,tr] =
train(net,inputs,targets,inputStates,layerStates);
% Test the Network
outputs = net(inputs,inputStates,layerStates);
errors = gsubtract(targets,outputs);
performance = perform(net,targets,outputs)
% View the Network
view(net)
%
%
%
%
%
%
%
%

Plots
Uncomment these lines to enable various plots.
figure, plotperform(tr)
figure, plottrainstate(tr)
figure, plotregression(targets,outputs)
figure, plotresponse(targets,outputs)
figure, ploterrcorr(errors)
figure, plotinerrcorr(inputs,errors)

% Closed Loop Network
% Use this network to do multi-step prediction.
% The function CLOSELOOP replaces the feedback input
with a direct
% connection from the outout layer.
netc = closeloop(net);
netc.name = [net.name ' - Closed Loop'];
view(netc)
[xc,xic,aic,tc] = preparets(netc,inputSeries,
{},targetSeries);
yc = netc(xc,xic,aic);
closedLoopPerformance = perform(netc,tc,yc)
% Early Prediction Network
% For some applications it helps to get the prediction
a
% timestep early.
% The original network returns predicted y(t+1) at the
same
% time it is given y(t+1).

% For some applications such as decision making, it
would
% help to have predicted y(t+1) once y(t) is
available, but
% before the actual y(t+1) occurs.
% The network can be made to return its output a
timestep early
% by removing one delay so that its minimal tap delay
is now
% 0 instead of 1. The new network returns the same
outputs as
% the original network, but outputs are shifted left
one timestep.
nets = removedelay(net);
nets.name = [net.name ' - Predict One Step Ahead'];
view(nets)
[xs,xis,ais,ts] = preparets(nets,inputSeries,
{},targetSeries);
ys = nets(xs,xis,ais);
earlyPredictPerformance = perform(nets,ts,ys)
You can save the script, and then run it from the command line to
reproduce the results of the previous GUI session. You can also edit the
script to customize the training process. In this case, follow each of the
steps in the script.
1.

The script assumes that the input vectors and target vectors are
already loaded into the workspace. If the data are not loaded, you can
load them as follows:

2. load ph_dataset
3. inputSeries = phInputs;
4. targetSeries = phTargets;
5.

Create a network. The NARX network, narxnet, is a feedforward
network with the default tan-sigmoid transfer function in the hidden layer
and linear transfer function in the output layer. This network has two
inputs. One is an external input, and the other is a feedback connection
from the network output. (After the network has been trained, this
feedback connection can be closed, as you will see at a later step.) For
each of these inputs, there is a tapped delay line to store previous
values. To assign the network architecture for a NARX network, you

must select the delays associated with each tapped delay line, and also
the number of hidden layer neurons. In the following steps, you assign
the input delays and the feedback delays to range from 1 to 4 and the
number of hidden neurons to be 10.
6. inputDelays = 1:4;
7. feedbackDelays = 1:4;
8. hiddenLayerSize = 10;
9. net =
narxnet(inputDelays,feedbackDelays,hiddenLayerSize);
Note Increasing the number of neurons and the number of delays requires
more computation, and this has a tendency to overfit the data when the
numbers are set too high, but it allows the network to solve more complicated
problems. More layers require more computation, but their use might result in
the network solving complex problems more efficiently. To use more than one
hidden layer, enter the hidden layer sizes as elements of an array in
the fitnet command.
10. Prepare the data for training. When training a network containing
tapped delay lines, it is necessary to fill the delays with initial values of
the inputs and outputs of the network. There is a toolbox command that
facilitates this process - preparets. This function has three input
arguments: the network, the input sequence and the target sequence.
The function returns the initial conditions that are needed to fill the
tapped delay lines in the network, and modified input and target
sequences, where the initial conditions have been removed. You can call
the function as follows:
11. [inputs,inputStates,layerStates,targets] = ...
12.
preparets(net,inputSeries,{},targetSeries);
13.

Set up the division of data.

14. net.divideParam.trainRatio = 70/100;
15. net.divideParam.valRatio

= 15/100;

16. net.divideParam.testRatio

= 15/100;

With these settings, the input vectors and target vectors will be randomly
divided, with 70% used for training, 15% for validation and 15% for
testing.
17. Train the network. The network uses the default LevenbergMarquardt algorithm (trainlm) for training. For problems in which
Levenberg-Marquardt does not produce as accurate results as desired,
or for large data problems, consider setting the network training function
to Bayesian Regularization (trainbr) or Scaled Conjugate Gradient
(trainscg), respectively, with either
18.
net.trainFcn = 'trainbr';
net.trainFcn = 'trainscg';
To train the network, enter:
[net,tr] =
train(net,inputs,targets,inputStates,layerStates);
During training, the following training window opens. This window
displays training progress and allows you to interrupt training at any
point by clicking Stop Training.

This training stopped when the validation error increased for six
iterations, which occurred at iteration 70.
19. Test the network. After the network has been trained, you can use it
to compute the network outputs. The following code calculates the
network outputs, errors and overall performance. Note that to simulate a
network with tapped delay lines, you need to assign the initial values for
these delayed signals. This is done
withinputStates and layerStates provided by preparets at an
earlier stage.
20. outputs = net(inputs,inputStates,layerStates);
21. errors = gsubtract(targets,outputs);
22. performance = perform(net,targets,outputs)
23. performance =
24.
25.

0.0042

26.
27.

View the network diagram.

28. view(net)

29.

Plot the performance training record to check for potential overfitting.

30. figure, plotperform(tr)

This figure shows that training, validation and testing errors all
decreased until iteration 64. It does not appear that any overfitting has
occurred, because neither testing nor validation error increased before
iteration 64.
All of the training is done in open loop (also called series-parallel
architecture), including the validation and testing steps. The typical
workflow is to fully create the network in open loop, and only when it has
been trained (which includes validation and testing steps) is it
transformed to closed loop for multistep-ahead prediction. Likewise,
the R values in the GUI are computed based on the open-loop training
results.
31. Close the loop on the NARX network. When the feedback loop is
open on the NARX network, it is performing a one-step-ahead
prediction. It is predicting the next value of y(t) from previous values
of y(t) and x(t). With the feedback loop closed, it can be used to perform
multi-step-ahead predictions. This is because predictions of y(t) will be
used in place of actual future values of y(t). The following commands
can be used to close the loop and calculate closed-loop performance

32. netc = closeloop(net);
33. netc.name = [net.name ' - Closed Loop'];
34. view(netc)
35. [xc,xic,aic,tc] = preparets(netc,inputSeries,
{},targetSeries);
36. yc = netc(xc,xic,aic);
37. perfc = perform(netc,tc,yc)
38. perfc =
39.
40.

2.8744

41.

42. Remove a delay from the network, to get the prediction one time step
early.
43. nets = removedelay(net);
44. nets.name = [net.name ' - Predict One Step Ahead'];
45. view(nets)
46. [xs,xis,ais,ts] = preparets(nets,inputSeries,
{},targetSeries);
47. ys = nets(xs,xis,ais);
48. earlyPredictPerformance = perform(nets,ts,ys)
49. earlyPredictPerformance =
50.
51.

0.0042

52.

From this figure, you can see that the network is identical to the previous
open-loop network, except that one delay has been removed from each
of the tapped delay lines. The output of the network is then y(t + 1)
instead of y(t). This may sometimes be helpful when a network is
deployed for certain applications.
If the network performance is not satisfactory, you could try any of these
approaches:


Reset the initial network weights and biases to new values
with init and train again (see "Initializing Weights"(init)).

Increase the number of hidden neurons or the number of delays.


Increase the number of training vectors.



Increase the number of input values, if more relevant information is
available.

Try a different training algorithm (see "Training Algorithms").
To get more experience in command-line operations, try some of these
tasks:


During training, open a plot window (such as the error correlation
plot), and watch it animate.

Plot from the command line with functions such
as plotresponse, ploterrcorr and plotperform. (For more
information on using these functions, see their reference pages.)
Also, see the advanced script for more options, when training from the
command line.
Each time a neural network is trained, can result in a different solution due
to different initial weight and bias values and different divisions of data into

training, validation, and test sets. As a result, different neural networks
trained on the same problem can give different outputs for the same input.
To ensure that a neural network of good accuracy has been found, retrain
several times.
There are several other techniques for improving upon initial solutions if
higher accuracy is desired. For more information, see Improve Neural
Network Generalization and Avoid Overfitting.

Sponsor Documents

Or use your account on DocShare.tips

Hide

Forgot your password?

Or register your new account on DocShare.tips

Hide

Lost your password? Please enter your email address. You will receive a link to create a new password.

Back to log-in

Close