Stocks & Commodities V. 2:2 (55-59): Optimizing the Maximum Entropy Method by Anthony W.

Warren, PhD

Optimizing the Maximum Entropy Method

by Anthony W. Warren, PhD

This article is the second in a series on the Maximum Entropy Method (MEM) and its use in technical
analysis for short data length spectrum estimation and short term forecasting. In the first article (January
1984 issue of T.A.) we introduced the main ideas behind MEM and illustrated the relationship between
the MEM order or number of days used in MEM forecasts, and the resolution in the spectrum. In the
following article we discuss the critical process of optimizing the MEM fit coefficients to obtain reliable
models for spectrum estimation. This process is crucial for obtaining high resolution spectra from short
data sets (say 30 to 80 samples of price history data), and for obtaining forecasts consistent with recent
data trends and cycle lengths. The optimizing methods discussed here represent the product of both
academic research on the Maximum Entropy Method and the author's research on application of this
method to stock and commodity data. The result of this effort is a semi-automated optimization algorithm
(to be published in a subsequent issue of T.A.) which is relatively easy to use, and which in most cases
will produce reliable spectra and forecasts. The application of this method to trend channel forecasting
and usage within trading systems will be discussed in a subsequent article.

MEM OPTIMIZATION PARAMETERS

There are three parameters which may be selected independently in the author's optimizing method:
* A prefilter-and-sample parameter- D
* The MEM prediction order- M
* The data length- N
The prefiltering method consists of applying a 2D length moving average to the user's data and
resampling the output every Dth point. (See the Appendix for a discussion on the prefiltering method.)
The net effect of this is to reduce the spectrum by D-fold, thus enhancing the signal spectrum and
reducing the noise power due to random daily price movements. Prefiltering is essential to obtain good
MEM spectra from short data length, noisy markets (as we shall illustrate shortly).
The MEM prediction order M is the number of samples (days) of prefiltered data used to make a forecast
or compute a MEM spectrum. Since the number of peaks in a MEM spectra is roughly equal to M/2, it is
important that M is not chosen too small or only low resolution, low quality spectra will be obtained. On
the other hand if M is chosen too large, then the spectrum estimates may be unreliable. (In our program,

Article Text Copyright (c) Technical Analysis Inc. 1

Stocks & Commodities V. 2:2 (55-59): Optimizing the Maximum Entropy Method by Anthony W. Warren, PhD

M cannot be selected larger than N+1/2 and in a commercially available MEM program, M is fixed at 12
days.)
The data length N directly influences the signal spectrum and should be (ultimately) chosen such that the
data length is at least two to three times the total number of days used to compute a forecast (N>=3*M*D
is recommended.) In principle one should use as much data as is relevant for the market of interest, since
the resolution and stability of the spectrum is dependent on the data length. On the other hand, if too
much weight is assigned to 'old' data trends then the data forecasts may be unreliable. With slower
moving stock and commodity markets it is inefficient and time consuming to use daily sampled data.
Thus, whenever more than, say 100 days of data are utilized, the data should be resampled at 3 day or
weekly intervals prior to MEM analysis.
In the author's MEM fit and spectrum analysis code, the user can either select D,M and N manually and
obtain a series of spectrum plots until satisfied with the results, or can select the data length N and have
the program automatically optimize the prefilter parameter D and the model order M. In the auto-mode
the program selects D values equal to 1,2,3,4 or 5, and then finds the MEM coefficients for all orders M
such that 1 < = M < = P+1/2, where P is the length of the prefiltered data. For each D and M value, a
Final Prediction Error (FPE) criteria is computed based on an average forecast error variance, and the
minimum FPE value over all M and D is selected. What this amounts to is that the optimization process
is done objectively, rather than relying on subjective 'feel' in selecting the correct model as is often the
case with model fitting techniques. (Ideally a trading method should handle most of the technical details
and free the trader to spend most of his time and judgment on trading decisions, rather than on optimizing
the trading parameters.)
The optimization method does not always produce a satisfactory spectrum or give D, M and N values
which truly minimize the FPE error criteria. However, this is to be expected since the data length N
initially selected may not be appropriate for current market conditions. Generally the FPE function
provides clues for reselecting the data length N, and for choosing alternate values of D and M.
Consequently, an option for printing FPE function values and printer plots at each MEM fit iteration is
available to provide a record of the optimization process.

MEM OPTIMIZER CASE HISTORY: PORK BELLIES

In the previous article on Maximum Entropy we analyzed the model selection process for a mutual fund
using over a year's worth of weekly sampled data. In this section we will illustrate the power of the
optimization method on a more difficult market. Pork Belly contract data was selected since this market
is very fast moving and has a high level of random trading noise in it's price movements. A number of
intermediate spectrum plots are used in the discussion in order to illustrate the optimization process and
the role that the FPE function plays. ( In using the code only the final optimized spectrum would
normally be displayed.)
Figure 1 shows the price history data used for the case study, about 80 days of Pork Belly closings over a
four month period. The data contains both a major upcycle and an erratic consolidation phase. If we
examine the Fourier power spectrum for this data shown in Figure 2, it is evident that most of the power
is in the lowest frequency peak corresponding to the main trend of the market. (The power spectrum is
shown in normalized frequency, i.e. cycles per sample period rather than in cycles per year since the
inverse of normalized frequency gives the number of samples (days) in the cycle of interest. For example,
the second major peak frequency =.07, which gives a cycle length = 1/.07 = 14 days .) Most of the power

Article Text Copyright (c) Technical Analysis Inc. 2

Stocks & Commodities V. 2:2 (55-59): Optimizing the Maximum Entropy Method by Anthony W. Warren, PhD

FIGURE 1

Copyright (c) Technical Analysis Inc.

Stocks & Commodities V. 2:2 (55-59): Optimizing the Maximum Entropy Method by Anthony W. Warren, PhD

FIGURE 2

FIGURE 3
Stocks & Commodities V. 2:2 (55-59): Optimizing the Maximum Entropy Method by Anthony W. Warren, PhD

spectrum, i.e. the frequencies above 0.1 are noise peaks which obscure the main signal frequencies of
interest. An ideal MEM spectrum for this data would contain just the two low frequency peaks below 0.1
cycles/day.
We now show the results of MEM model fitting and order selection with no preprocessing. The first 40
days of data shown in Figure 1 are used for this and the subsequent analysis. (It was necessary to use 80
days in the Fourier analysis in order to resolve the second peak frequency at about .07 cycles/day.) Figure
3 shows the FPE function for MEM fits up to order +20. The minimum FPE for model selection is at
order 3, although there is a second local minimum at order 9 in this case. Figure 4 shows a plot of the
MEM spectrum for the 3rd order model fit. The plot is clearly lacking in resolution, and is marginal for
selection of optimum moving averages or data forecasting. This example illustrates the dilemma involved
in using Maximum Entropy on noisy data without preprocessing, i.e. the analysis method cannot resolve
the spectral peaks of interest and results in selecting the prediction order M too small. (Even if we select
the second local minimum at order 9, the signal spectrum lacks resolution in the frequencies of interest.)
We now illustrate the application of the author's MEM optimizer to this data:
At the first fit iteration (D=1) the data is preprocessed by computing a two point moving average. The
output of the MEM program at the end of this iteration is the FPE printer plot shown in Figure 5. The
FPE in this case shows a clear minimum at order 8, and is designated with a '+' in the printer plot. If we
plot the MEM spectra corresponding to this case, we obtain Figure 6. This spectrum is clearly better than
that of figure 4, but is still insufficient to resolve a second spectral peak below 0.1 cycles/day.
At the second fit iteration (D=2) the data is smoothed with a 4 day moving average and resampled in 2
day intervals. The FPE output at the end of this iteration is shown in Figure 7. (Actually the FPE values
as defined in the earlier paper have been normalized by dividing by D and multiplying by 100. It is
evident that there is a local minimum at order 4 and a true minimum at order 10 which is the largest
allowed prediction order, since there are only 19 smoothed and resampled values. Note also that the
minimum FPE value is significantly less than that for Figure 5, which indicates an improvement in the
model fit process. If we plot the MEM spectra corresponding to the minimum values, we obtain Figures 8
and 9. (These spectra are only defined out to half the frequency band of the previous spectra, since
resampling by D days reduces the spectrum D-fold.) The 4th order spectra suffers from lack of resolution
as in the earlier fits, whereas the 10th order spectrum is able to resolve all the spectral peaks of interest.
Unfortunately, the very sharp spectral peaks in Figure 9 are evidence of model over-resolution, which is
common when the order selected is the maximum allowed.
At the third fit iteration (D=3) the minimum FPE occurs at order 6, and the FPE value is greatly reduced
compared with the earlier cases. (See the summary table below.) This shows that the model fit is much
better than before. Figure 10 shows the corresponding MEM spectra. This spectrum nicely resolves the
two main spectral peaks and the noise peaks at frequencies exceeding .12 cycles/day have been
eliminated by the prefiltering method. The spectrum is slightly over-resolved in this case also, since the
order selected is the maximum allowed for the number of prefiltered data values.
At the fourth fit iteration the data is smoothed with an 8 day moving average and the minimum FPE
occurs at order 4. The FPE value is again lower than in the previous cases. This case turns out to be the
optimum value selected by the algorithm, since the fifth and last fit iteration yields a min FPE value
much larger than that for D=4. (See the table below.) The spectrum plot for the optimized MEM fit is
shown in Figure 11. This plot nicely resolves the two signal spectrum peaks and shows no evidence of

Article Text Copyright (c) Technical Analysis Inc. 3

Stocks & Commodities V. 2:2 (55-59): Optimizing the Maximum Entropy Method by Anthony W. Warren, PhD

FIGURE 4

FIGURE 5
Stocks & Commodities V. 2:2 (55-59): Optimizing the Maximum Entropy Method by Anthony W. Warren, PhD

FIGURE 6

FIGURE 7
Stocks & Commodities V. 2:2 (55-59): Optimizing the Maximum Entropy Method by Anthony W. Warren, PhD

FIGURE 8

FIGURE 9
Stocks & Commodities V. 2:2 (55-59): Optimizing the Maximum Entropy Method by Anthony W. Warren, PhD

FIGURE 10

FIGURE 11
Stocks & Commodities V. 2:2 (55-59): Optimizing the Maximum Entropy Method by Anthony W. Warren, PhD

model overfitting. Consequently, the model optimizer in this case selects the same parameter values that
a technical analyst would select based on intensive analysis of the fit spectra. The optimum moving
average for the spectra of Figure 11 also has length = 8 days, based on halving the cycle length of the
highest significant frequency component.

MEM CASE STUDY SUMMARY TABLE

SUMMARY
One of the problems in applying spectrum analysis to stock and commodity data is that these markets are
generally non-stationary, i.e. the dominant trends and cycles evolve and change phase over time. Fourier
Analysis in particular requires long data lengths to provide adequate spectrum resolution of the dominant
cycles for selection of moving average lengths and other trading parameters. MEM spectrum analysis and
forecasting is very promising for application to stock and commodity trading since it is possible to
identify the dominant cyclic components using short data lengths. However, data prefiltering is generally
required with this method in order to obtain good spectrum resolution in high noise level markets.
An important breakthrough in applying the Maximum Entropy Method was the development of objective
order selection techniques. Higher performance and reliability could be obtained by optimizing the model
order rather than using fixed order models. Moreover, objective order criteria are simpler to use than
pattern recognition or statistical tests which require expert analysis and judgment. The author has
extended the FPE order selection method to optimize the model order and prefiltering parameters. This
technique makes it feasible to obtain high resolution MEM spectra and quality data forecasts from noisy
stock and commodity data.

APPENDIX: MEM PREFILTERING METHOD

Conceptually, the author's prefiltering method consists of a data smoother applied to the input data,
followed by a resampling operation. The key element in this method is the resampling operation which
reduces the power spectrum frequency band prior to maximum entropy analysis. The data smoother is
primarily used to reduce the high frequency noise components prior to resampling. (See p. 25 of the Nov.
1983 article 'Fourier Analysis, Faster and Better'.) Originally the author used the prefiltering method
discussed in the earlier articles, i.e. averaging the data in groups of M points, which is equivalent to
applying an M point moving average followed by M point resampling. However, in a significant number
of test cases, this method produces an artificial spectrum peak near the high frequency endpoint of the

Article Text Copyright (c) Technical Analysis Inc. 4

Stocks & Commodities V. 2:2 (55-59): Optimizing the Maximum Entropy Method by Anthony W. Warren, PhD

MEM spectra. This is due to a phenomenon called aliasing, wherein frequency peaks that are outside the
reduced spectrum interval are 'folded' back into the resampled spectrum. The use of a 2M moving
average followed by M point resampling greatly reduces the aliasing problem, since this method
effectively nulls out the power spectrum near the maximum or folding frequency. (See Figures 8 to 11.)
This prefiltering method works quite well as long as the signal peaks of interest are in the lower half of
the reduced spectrum. Consequently, this method was selected for the author's MEM code.
Anthony W. Warren, Ph.D. is an applied mathematics consultant for Boeing Computer Services
Company. Dr. Warren's background and security interests make him particularly well suited to develop
mathematically sound computer applications.

Figures Copyright (c) Technical Analysis Inc. 5