C 2012 Public Financial Publications, Inc.

The Politics of Forecast Bias: Forecaster Effect

and Other Effects in New York City Revenue
Forecasting
DANIEL W. WILLIAMS

This paper examines the impact of forecasters, horizons, revenue categories, and
forecast timing in relation to decision making on forecast bias or accuracy. The
significant findings are: for the most part forecasters tend to report forecasts that
are similar rather than competitive. Forecast bias (underforecasting) increases over
longer horizons; consequently claims of structural budget deficit are suspect, as
an assertion of structural deficit requires that a reliable forecast of revenue shows
continuous shortfall compared with a reliable forecast of expenditures. There is
an overforecasting bias in property tax, possibly reflecting demand for services.
There is an underforecasting forecast bias in two revenue categories, all other
taxes and federal categorical grants, resulting in a net total underforecasting bias
for the city’s revenue. There appears to be a period effect (forecasts in June are
substantially biased), but this effect requires further study. The study suggests
further examination of the bias associated with revenue categories, time within the
budget cycle, and forecast horizon.

INTRODUCTION

As long ago as Government Budgeting (Burkhead 1956), it has been observed that gov-
ernment forecasters often underestimate revenue. In fact, New York City consistently un-
derforecasts revenue with forecasts in the earlier years—sometimes roughly accurate, but
almost never in excess of actual revenues. Table 1 also shows that in the past decade, un-
derforecasting has been pervasive across revenue sources. Treating New York City as a
forecasting case study, this paper examines forecast details to discover characteristics that
predict underforecasting bias.
There have been many studies substantiating the notion of revenue underforecasting
by subnational governments in the United States. Voorhees (2006) provides an extensive

Daniel W. Williams is a Associate Professor, Baruch College, New York, NY 10010. He can be reached at
Daniel.Williams@Baruch.CUNY.EDU. I would like to thank Fred Thompson, Phil Joyce, Jonathan Justice
and Thad Calabrese for comments on prior versions of this paper.

Williams / The Politics of Forecast Bias 1

 TABLE 1
End of Year Revenue Variance from Beginning of Year Budgeted Revenue (Forecast as
Accepted in the Appropriation Process) and Proportionate Shares of Revenue
FY Taxes All own source Federal and State aid All revenue
2001 6.2% 9.2% 5.0% 7.8%
2002 −4.3% 6.5% 13.7% 3.8%
2003 2.0% −5.4% 15.2% 5.5%
2004 8.5% 8.1% 11.2% 9.1%
2005 14.1% 12.7% 11.2% 12.7%
2006 12.6% 12.2% 0.2% 8.4%
2007 16.4% 15.9% 2.9% 11.9%
2008 5.8% 4.9% 5.0% 5.9%
2009 −1.2% −2.5% 6.8% 2.5%
2010 5.7% 9.3% 6.2% 6.5%
2011 3.7% 5.7% 5.0% 4.5%
All years 6.3% 7.0% 7.5% 7.1%
Proportions 50.9%∗ 10.9% 28.8%** 18.0%
∗ Taxes:22.7% property tax, 38.2% all other. ** Federal 10.8%, State 18.0%.
Source: Compiled by the author from the NYC Comprehensive Annual Financial Reports 2001–2011.
Source of Proportions: Compiled by the author from reports from the New York City Office of Management and
Budget based on average revenue over Fiscal Years 2005 through 2009.

review of the recent literature, reporting that underestimation bias varies substantially from
study to study. Rubin (1987) reports that poor jurisdictions tend to overestimate revenue.
In one study comparing the US Office of Management and Budget and Congressional
Budget Office, it was determined that competing forecasters exhibit similar biases rather
than providing competing independent forecasts (Krause and Douglas 2006).
Choate and Thompson (1988, 1990), Paleologou (2005), and Rodgers and Joyce (1996)
focus on the political character of forecasting. Choate and Thompson argue that it is the
principal (the political decision maker), not the agent (the technical forecaster), who selects
the underforecast. They argue that forecast bias is not strictly a consequence of the widely
believed budget officers’ risk aversion, and suggest that underforecasting plays a role in tax
policy. Paleologou shows that in the United Kingdom forecasting bias is associated with
the party in power. Rodgers and Joyce suggest there are complex factors some of which are
rational in a political context and they call for further work to explain variation in forecast
errors.
This paper examines revenue forecasting differently than past studies. First, rather than
looking at one, or even two, revenue forecasters, this paper examines five separate forecast-
ers of New York City revenue over a five-year period. By examining a larger number of
forecasters, it is possible to evaluate whether the institutional motivation and presence of
competition affects revenue forecasting accuracy and the direction of bias.

2 Public Budgeting & Finance / Winter 2012

 Second, the paper examines whether there are differences in forecasting accuracy by
revenue types and forecast timing. Revenue categories might be associated with bias because
they have separate political principals, and, from a more technical perspective, some are
easier to forecast than others. Budget cycle timing could be related to bias because bias may
play a beneficial role in some stages of budget negotiation.
Typically, studies of forecast bias are excessively high level, providing little insight into
when and where forecast bias actually arises. This study takes an alternative approach look-
ing at substantial detail to determine where bias occurs. Forecast bias cannot be ameliorated
when such details are unknown. The questions asked below reflect many well-known sources
of real world forecast concerns.
The specific questions that will be examined in this paper are:

1. Is there a forecaster effect on revenue forecasting? Revenue conservers may be more

interested in underforecasting while program advocates may prefer overforecasting.
2. Is there a revenue category effect on revenue forecasting? There are at least two reasons
for revenue category effect. First, some sources of revenue are more easily forecast
than others. For example, property tax is subject to only a small uncertainty related
to noncompliance and an even smaller uncertainty related to net change in taxable
property. Second, some taxes may be less palatable to the electorate and may, therefore,
lead to behavioral effects for forecasters or their managers.
3. Is there a forecast horizon effect on revenue forecasting? It is well known that forecast
accuracy diminishes over time (Makridakis et al. 1982, 1993; Makridakis and Hibon
2000; Makridakis, Hibon, and Moser 1979). If forecasts are biased, the effect size of the
bias should, likewise, increase over time. This possibility is of particular significance
related to forecasts that may be used in discussions of structural balance.
4. Is there an effect related to the month of origin of the revenue forecast? The month
of origin is related to the political cycle. As the budget calendar progresses, concrete
commitments resultant from the forecast become hardened. Thus, later period bias
is more difficult to reverse. If bias is a deliberate political ploy, then early forecasts
should be more biased than later forecasts.
5. Is there an effect related to repetition of the revenue forecast in the same cycle? As
with the previous question, repeat forecasting may allow for early period bias, which
may only appear with forecasters who offer multiple forecasts in the same cycle.

These questions relate directly to the reliability and use of forecasts and the need for
forecasts, as well as addressing significant gaps that exist in the current research. In the
subsequent sections, each question is discussed in more detail.
This paper is significant because revenue forecasting is a principal controlling force in
budget making in subnational governments in the United States. Elected officials of all sorts
are loath to be seen to increase taxes or any visible revenue devices. Forecasting provides
estimates for the continuing effect of already established revenue devices for the budget
year and future years. Elected officials may constrain expenditures within these estimates

Williams / The Politics of Forecast Bias 3

and may, in some instances, change the direction of future year expenditures because of
these forecasts (Crain and Muris 1995; Garman, Haggard, and Willis 2001; Heller 1997;
MacManus and Pammer 1990; Pressman 2004; Rose 1985; Weaver 1986).

Is There a Forecaster Effect?

Jurisdictions of substantial size sometimes have multiple revenue forecasts made by distinc-
tive forecast units such as a legislative unit and an executive unit; but the resultant forecasts
may not be treated as competitive. Instead they may be used as inputs for a consensus fore-
cast, a common device for reconciling forecasts.1 This consensus forecast may be the only
publically released forecast. Where there are competitive forecasts, the publically available
data may be limited.
In New York City there are many government sponsored forecasts, but no consensus
forecast. The New York City Charter provides that the mayor’s forecast determines the
revenue estimate of all revenue sources except property tax, which the city council can
adjust by adjusting the tax rate. So, consensus forecasting is not required by the political
process. This provides an opportunity to examine forecast bias.
The theory of bias proposed here is that political decision makers and senior management
prefer bias and communicate this preference to forecasters, not that the particular algorithms
used by forecasters are more susceptible to bias than alternatives not used. All the New
York City forecasters use essentially the same sorts of models, primarily regression or
ARIMA type econometric models; the exception is the property tax, which can be computed
deterministically. Some lesser revenue categories may be estimated with simpler models.
All forecasters examined here forecast the entire city budget. Aggregation of categories
(discussed later) is completed to make the forecasts parallel.
The forecasters include:

1. New York City Office of Management and Budget (NYCOMB), which reports to a
deputy mayor and ultimately to the mayor. This is the charter mandated determinative
forecast.
2. The Financial Control Board (FCB), which was created after the financial crisis of
the 1970s. The explicit purpose of the FCB is to prevent the city from overspending
its revenue.
3. The New York City Comptroller (NYCC), one of three citywide elective positions
and is considered a competitor of the mayor. In the 2009 election and other earlier
elections the comptroller was among the mayoral candidates, sometimes opposed to
the current mayor.
4. The Independent Budget Office (IBO), which was created in a significant charter
revision in 1989. This charter change resulted in shifting budget power to the city

1. The author served on a technical panel for a consensus prison population forecast between three bodies
when employed with a state government.

4 Public Budgeting & Finance / Winter 2012

 council and the IBO was created as technical advisory staff to the city council and to
the public. As the city council does not publish a forecast, the IBO forecast is the best
available proxy for the city council forecast.
5. New York State Deputy Comptroller for New York City (DCNYC) represents the
state’s interest and in some ways has a similar function to the FCB. Half of the state’s
population lives within New York City and far more than half of the state’s revenue is
derived from activity within New York City, so the state comptroller is also engaged
in maintaining oversight of the city revenue estimates.

During recent years each of these entities has made some information about its fore-
cast available to the public via the internet.2 These publically available data provide the
opportunity to examine relative forecast practice of the five forecasters.
Should there be a forecaster effect? By forecaster effect, it is intended that there is a
statistically significant observable difference in systematic error associated with the source
of the forecast (the entity that produces the forecast). There are three reasons why there
might be such a difference:

1. Among the five forecasters, two, NYCOMB and FCB, are revenue conservers (Bland
2007)—if there is a bias for underforecasting, they should exhibit this bias the most;
two, IBO and NYCC, represent demand for services—IBO as a proxy for city council
and NYCC as a political competitor to the mayor—so they should reflect less under-
forecasting bias; and one, DCNYC, is indeterminate. This reason is the most explicitly
political: revenue conservers such as NYCOMB and the FCB may be seeking to sup-
press expenditures in order to build surpluses as a hedge against future uncertainty or
to hold down future taxes as argued by Choate and Thompson (1988, 1990). Repre-
sentatives of demand, such as IBO and NYCC, should prefer to find revenue so that
expenditure programs can be funded (Bretschneider, Straussman, and Mullins 1988).
2. It is not an effective use of public resources to conduct five forecasts that get the
same biased answer. So, forecasters should be expected to find different answers. The
existence of no forecaster effect should raise the suspicion that the public is spending
too much money on forecasting. The existence of rational bias is well documented
in forecasting literature (Batchelor 2007; Butler and Lang 1991; Laster, Bennett, and
Geoum 1999). The specific rational bias suggested here, coming to different results,
is sometimes included in this literature, although it is poorly understood. The reason
offered here—the continued appearance of usefulness—is no more speculative than
those offered in the existing literature.

2. These websites include: NYCOMB budget publications at http://www.nyc.gov/html/omb/html/

publications/publications.shtml, FCB at http://www.fcb.state.ny.us/, IBO at http://www.ibo.nyc.ny.
us/, NYCC at http://www.comptroller.nyc.gov/bureaus/bud/, and DCNYC at http://www.osc.state.
ny.us/osdc/index.htm. These websites are revised rather frequently, so these URLs are to top level com-
ponents of these websites. Actual data and reports are within these sites.

Williams / The Politics of Forecast Bias 5

 3. The forecasters may have differing levels of access to contextual information about
different components of their forecast. For example, DCNYC may have more insight
into the funding of state categorical grants to the city. Such information advantages
should result in some systematic differences in the forecasts.

There are also reasons why there should be no forecaster effect.

1. As argued by Krause and Douglas (2006), forecasters may seek the same answer rather
than differences. This result is a safe strategy because the future is uncertain, leaving
opportunity for variance between forecast and actual. When the same variance can be
attributed to one’s “competitor,” it appears one is still doing a reasonably good job.
Herding behavior has been documented in other uses of forecasting as well (Clement
and Tse 2005; Olsen 1996).
2. To the degree that it is documented on their various websites and reports, these entities
use the same basic approach to forecasting, mostly econometric modeling. To some
degree they are simply second guessing NYCOMB’s parameter choices. NYCOMB
produces a set of input economic factor forecasts, which are not necessarily reforecast
by the other entities. In addition, for “minor” categories various forecasters simply
accept the NYCOMB forecast as their own.3 Considering the similarity of method and
high interdependence, it is unlikely that the forecasts will be substantially different.
3. Some data deficiencies that will be discussed in a subsequent section led to imputing
some values. Thus, even where there may be differences, the data may be insufficient
to find these differences.

Hypothesis 1 = There is a forecaster effect that is biased to underforecasting (negative

coefficients) for NYCOMB and FCB, neutral or biased to over forecasting
(positive coefficients) for IBO and NYCC, and uncertain for DCNYC.
Because of the conflicting expectations, there is no clear expectation that
the data will be consistent with this hypothesis.

Is There a Revenue Category Effect on Forecasting?

Five revenue categories are examined, property tax, all other taxes, miscellaneous and other
city revenue, federal categorical grants, and state categorical grants. This selection is made
as a compromise between excessive aggregation and excessive use of imputation as some
forecasters do not report in even this detail for all years examined.
Table 1 shows that property tax is 22.7 percent of all revenue, the second largest category.
It is of particular interest because city council has a role in setting the forecast by changing

3. No interviews were conducted with the various forecast entities. These entities were contacted for
data files that were not available on their websites. In the ensuing conversation some anecdotal information
was volunteered. This information should not be treated as systematically collected. Despite the contacts, no
additional data were provided beyond that which could be found on the websites.

6 Public Budgeting & Finance / Winter 2012

the tax rate. Also, property tax is especially simple to forecast as it is tax rate times base (ad-
justed for properties coming on or going off the tax roll and updated for current assessment)
times compliance rate (all by category). Although there is a small amount of uncertainty
in tax base and compliance rate, within the existing stock of New York property, only the
compliance rate can change more than a tiny amount. Because it is hard to attribute vari-
ance in property tax forecast to technical difficulties and because it is controlled, in part, by
representatives of demand, the budget year forecast of property tax should either have no
bias or positive (over forecasting) bias.
Table 1 shows that all other taxes is the largest category of revenue at 38.2 percent. It
is a residual of all taxes minus property taxes and includes personal income tax, general
corporation tax, banking corporation tax, unincorporated business tax, sale and use tax,
commercial rent tax, real property transfer tax, mortgage recording tax, utility tax, cigarette
tax, hotel tax, all other tax, and tax audit revenue. It would be desirable to compare the
forecasts at the line level, but some forecasters do not even provide the simple distinction
between property taxes and all other. To minimize the need for imputing values, only
the two tax categories are included. Because the forecast of this category is controlled by
NYCOMB, a revenue conserver, and can involve many difficult forecasts, it is anticipated
that this category will show an underforecasting (negative) bias.
Table 1 shows that miscellaneous and other city revenue is the second smallest category
at 10.9 percent. It is an aggregate of two reported categories: (1) miscellaneous, where the
city reports most such own source revenue as licenses, fines, interest earnings, and so forth.
This category is clearly reported by nearly all forecasters for nearly all years. (2) All other
city revenue, where the city reports any other revenue except federal and state categorical
grants. This category is quite small.4 As the forecasts of these categories are controlled by
NYCOMB, it is anticipated that this category will show an underforecasting (negative) bias.
However, this anticipation is somewhat uncertain.
The smallest revenue category reported in Table 1 is federal categorical grants, at
10.2 percent. This category refers to funds in four federal grant categories: Community
Development, Social Services, Education, and other. New York City estimates how much it
will receive in these grants categories. The city’s interest in forecasting these grants is mixed.
Anticipating the receipt of these grants may lead the city to commit to expenditures that
it may later find itself having to fund with local revenue. However, that is not always the
case. Some of these grants may serve as simple pass-throughs or other devices that do not
commit the city to anything in particular should they fail to materialize; failing to anticipate
receipt of such funds would have the effect of reducing the probability of the overall social
well-being of the city. Underforecasting of federal grants could lead the grantor entities
to believe the city does not really need the funds requested in grant applications; however,

4. All other city revenue includes federal and state unrestricted grants, other categorical grants interfund
agreements, disallowance of categorical grants (a negative entry), and adjustment for intercity revenue (a
negative entry, which is a revenue line in miscellaneous category). It is apparent that the standard “forecast”
for disallowance of categorical grants is set at $15 million every year. By combining the two larger categories,
intercity revenue is netted out of the entire forecast and actuals, thus reducing irrelevant variability.

Williams / The Politics of Forecast Bias 7

mitigating against this is the difficulty of tracking from particular grants to the city revenue
forecast because of the aggregation of lines to the forecast and differences in fiscal years
and the fact that many, perhaps most, of the funding is supplied through formula grants
that have little to do with city requests. Due to the mixed nature of these concerns, there is
no clear direction to any possible bias in the revenue forecast.
State categorical grants account for the remaining 18 percent of the revenue. This category
includes funds in Social Services, Education, Higher Education, Department of Health and
Mental Hygiene, and other. New York City estimates how much it will receive in these
grants categories. Considerations with state grants are similar to those of federal grants,
thus there is no clear direction to any possible bias in the revenue forecast.

Hypothesis 2 = There is a revenue effect that is negative for all city source categories
except property (which is uncertain) and uncertain for federal and state
categorical grants.

Is There a Forecast Horizon Effect on Forecasting?

Almost universally, previous research has found that forecast accuracy deteriorates over
time (Fildes et al. 1998; Gardner and McKenzie 1985; Makridakis et al. 1993; Makridakis
and Hibon 2000; Smith and Sincich 1991). New York City’s forecasts are for next year and
three out years. As the initial hypothesis is that the forecast is biased, the out years should
have a negative coefficient that reflects the effect size of the bias over time. The possibility
of a forecast bias in out years is of substantial interest. Since fiscal crisis of the 1970s, New
York City has existed under a perpetual cloud of “structural deficit,” that is, the expectation
that future year expenditure obligations are systematically higher than future year revenues
(Chen and Barbaro 2008; Cooper 2004; Levy 1996; Rohatyn 1994; Shefter 1992). If there
is an out year revenue forecast bias, this structural deficit may be illusory, although this
cannot be fully determined without also examining the expenditure forecasts. If, as asserted
at the beginning of this paper, these revenue forecasts are biased toward underforecasting
and if forecast accuracy deteriorates over time, it can be expected that the bias will become
more pronounced over time.

Hypothesis 3 = Underforecasting (negative) of revenue is predicted to increase with the

forecast horizon.

Is There an Effect Related to the Month of Origin of the Forecast?

The forecasts examined in this study are reported in May, June, and July of each year. May
and some June forecasts are related to the budget request. July and other June forecasts
relate to the final revenue estimates before city council makes its final budget decision. If,
as it is sometimes believed, the mayor holds back some revenue for budget negotiations

8 Public Budgeting & Finance / Winter 2012

with the city council, there may be a positive effect (comparative positive bias) for forecasts
made later in the process.

Hypothesis 4 = There is a month of origin effect that is negative (underforecast) for May,
uncertain for June, and positive for July (increase over earlier periods).

Is There an Effect Related to Repetition of Forecast Reporting in the Same Cycle?

Closely related to month of origin is the issuance of two forecasts in the same cycle. The
first forecast, reported in May or June, is the opening offer in the budget negotiation. The
second forecast reflects postnegotiation adjustments and may have a positive effect (relative
increase).

Hypothesis 5 = The second forecast for the same fiscal years is predicted to be higher.

Data
The data used in this analysis are collected from reports posted on the websites as identified
in footnote 2. The reports require considerable preanalysis before they can be examined
with respect to the central question of this study. NYCOMB and IBO report their forecasts.
However, these forecasts are divided into two parts. The larger part of these forecasts is
in revenue categories, such as property tax, sales tax, etc. A smaller part is in proposals
that are expected to pass at the city council session at the time of the forecast. Forecasts
of these proposals must be recategorized to fit the revenue categories used in this study
before comparison with actual revenue outcomes.5 After these adjustments and in order to
be parallel with the other forecasters, the NYCOMB forecast is summarized in five groups,
property tax, all other taxes, miscellaneous and other city revenue, federal categorical grants,
and state categorical grants. A further difficulty with the IBO forecast is that in the first year
of these data, 2003, IBO aggregates federal and state categorical grants. To impute values,
the NYCOMB value federal categorical grants is imputed to IBO and the difference between
the aggregate values and the NYCOMB values are treated as the IBO value state categorical
grants. This procedure may have the effect of understating the difference between IBO and
NYCOMB in federal categorical grants while overstating it in state categorical grants.
The NYCC, FCB, and DCNYC report “risks,” which is to say how much they think NY-
COMB’s forecast is incorrect in particular lines. These risks are converted back to forecasts
by adding or subtracting them compared with the appropriate NYCOMB forecast.6 These
forecasters also do not necessarily report all the categories of this analysis. Particularly, the

5. There is a report made by NYCOMB that assigns proposal forecasts to revenue categories; however,
efforts to obtain this report were unsuccessful. Most proposals are easily categorized; however, there may be
some small error due to this lack of access.
6. Despite the term “risk,” these are alternate competing forecasts. The “risk” is the difference between
the NYCOMB’s forecast and competing forecast.

Williams / The Politics of Forecast Bias 9

FCB and DCNYC do not always report property taxes, so the aggregate of all taxes must
be decomposed following the same procedure as described with IBO above. The variance
between NYCOMB and these other forecasters may be understated with respect to property
tax while overstated with respect to all other taxes. While this is unfortunate, property tax is
a special case as discussed above and it is desirable to perform some separate analysis even
if imperfect.
The forecasts examined in this study were made in Fiscal Years 2003 through 2007, while
preparing the budget for Fiscal Years 2004 through 2008. While one forecast produced by
the NYCC showed only a one year horizon, all the others showed a four year horizon. Thus,
with the one exception, a forecast prepared in Fiscal Year 2003 had projections for Fiscal
Years 2004 through 2007, and so forth. Actual revenues are available for Fiscal Years 2004
through 2009.
Each specific variable is addressed in the Appendix.7

Model
The hypotheses are evaluated using the model:
⎛ ⎞
⎜ F −A⎟ k l
⎝ F + A ⎠ × 100 = βi xi β j+k m j + ε
i=1 j=1
2
where F is forecast, A is actual, the variables xi are dummies for conditions specified in the
hypotheses except that the horizon variable is a semicontinuous variable labeled Out Year
and coded 0 through 3 for the budget year and three out years reported with the forecasts,
and mj are three-way or four-way interaction variables included as controls plus controls
for forecast and expenditure years.8 The expression,
F −A
× 100
F +A 2

is the symmetrical percent error (SPE), a component of symmetrical mean absolute percent
error (SMAPE), which is commonly used by forecasters (Armstrong 2006; Dekker, van
Donselaar, and Ouwehand 2004; Lawrence, O’Connor, and Edmundson 2000; Makridakis
and Hibon 2000). Use of SPE simultaneously solves four problems. As with differencing,

7. The dependent variable observations for this study are symmetrical percent errors and, for some
purposes, percent errors. For each actual expenditure within a category, there are many forecasts resulting in
roughly 20 observed symmetrical percent errors: There are at minimum five forecasters times four forecasts
(beginning with the third out year and some years later as the budget year). When a forecaster reports more
than one forecast in the same cycle, there can be more.
8. The forecast year is the fiscal year in which the forecast is made, the forecast origin year. The expenditure
year is the fiscal year in which the expenditures will occur, the forecast target year. A forecast made in FY 2003
will have target years of FY 2004 through FY 2007.

10 Public Budgeting & Finance / Winter 2012

SPE resolves problems related to nonstationarity of the errors. SPE shrinks if the size of the
error remains the same but the base increases, which is appropriate because the significance
of the error diminishes. Second, unlike with differencing, this variable does not reduce the
number of observations to correct for the problem of nonstationarity. Third, SPE retains
the sign, which is important for examining the direction of bias. Fourth, when all forecast
and actual values are positive, SPE value is confined to the range ±200 percent, which
reduces the influence of large outliers.9
The variables ki=1 βi xi test the hypotheses that the categorical variables coded as in-
dictor variables are biased. These variables include Out Year, Second, and blocks for the
revenue categories, forecaster, and month of forecast. The coefficient measures the additive
component of bias and can be interpreted directly.10
The main control variables lj=1 β j+k m j are indicators for forecast origin and fore-
cast target (expenditure) years.11 In addition there are four-way interaction dummies for
forecaster, revenue category, forecast month, and forecast year interacted. The three-way
interactions are triplet sets of these excluding interaction with the variable in the full set
consider in the main model. The interactions are expected because the primary control
for the forecast origin can be expected to have crossover relationships with all the other
variables; that is, any forecast organization may change the way it forecasts any particular
revenue category at any arbitrary time and may either retain or abandon that change.

Results
The mean value of SPE is −0.127 (12.7 percent) and the standard deviation is 13.4 percent.12
Regression estimations are shown in Table 2. The models are estimated using ordinary least
squares (OLS) with robust standard errors.
Models 1 through 3 test various hypotheses. Adjusted R-squared ranges from 50.7 to
68.0 percent, showing that, in fact, these variables provide a reasonably good explanation
of the variation in SPE. To test the robustness, Model 4 examines the effect of a reduced
form model excluding interaction variables. The adjusted R-squared declines by about
10 percent, but remains significant at the 1 percent level. The coefficients do not change
more than one or two percent points.

9. Small values of SPE, between ±10 percent, are roughly equivalent to percentages; large values under-
state positive percentages and overstate negative ones, but not severely until SPE exceeds ±25 percent.
10. As a technical note, by excluding the constant, the model allows one block of indictors to be included
without excluding the last indicator to avoid singularity. For this block, the coefficient measures the distance
from zero. For the other blocks, the coefficient measures the distance from the excluded indicator. In effect,
the constant is allocated among the indicators of the fully included block.
11. The first expenditure year and last forecast year are the excluded indicators. In addition, the model
fitting excluded the last forecast target year. Further explanation is provided in footnote 8.
12. Because this value is unweighted, it is not the mean bias.

Williams / The Politics of Forecast Bias 11

 TABLE 2
Dependent Variable: Symmetrical Percent Error
Forecaster Revenue Forecast month Reduced
Model 1 Model 2 Model 3 Model 4

Coef. Std. err. Coef. Std. err. Coef. Std. err. Coef. Std. err.
Out Year −0.039 0.004b −0.039 0.004b −0.040 0.004b −0.039 0.004b
Second −0.017 0.016 −0.009 0.039 −0.012 0.022 −0.016 0.017
NYCC 0.003 0.018 0.009 0.023
IBO 0.016 0.024 0.024 0.029
DCNYC −0.011 0.016 −0.003 0.021
FCB −0.013 0.009c −0.007 0.016
NYCOMB 0.000 0.024 0.009 0.029
Property 0.133 0.027a 0.146 0.008a
All other taxes −0.112 0.029b −0.122 0.011b
Miscellaneous −0.009 0.028 −0.011 0.011
Federal categorical −0.038 0.027c −0.039 0.009b
grants
State categorical 0.002 0.027
grants
May −0.009 0.010 −0.011 0.022
June −0.017 0.020 −0.011 0.015
July −0.002 0.035
R-squared 0.723 0.543 0.701 0.588
Adjusted R-squared 0.680 0.507 0.623 0.575
F 67.9 (84,540) 54.8 (46,578) 25.0 (157,593) 129.8 (20,604)
p-value (F) 0.000 0.000 0.000 0.000
Note: Control (interaction) variables not shown. All models controlled for expenditure year and budget year fixed
effects. Model 1: parameters estimated for forecaster effect. Model 2: parameters estimated for revenue category effect.
Miscellaneous and other city revenue; and federal categorical grants have an unexpected sign. Model 3: parameters
estimated for forecast month effect. Model 4 is reduced using only forecast origin and forecast target controls.
Significance: a 1%, two tail; b 1% one tail; c 10%, one tail.

Testing the Hypotheses

Hypothesis 1 is that there is a forecaster effect that is biased to underforecasting (negative
coefficients) for NYCOMB, FCB neutral or biased over forecasting (positive coefficients)
for IBO and NYCC, and uncertain for DCNYC. Model 1 shows negative effects for FCB
and DCNYC, positive effects for IBO and NYCC, and a coefficient roughly equal to
zero for NYCOMB. However, only the effect for FCB is statistically significant (at the
10 percent level). This result is consistent with the view that forecasters tend to agree rather
than compete. However, Model 1 examines whether any of these forecaster’s bias can be

12 Public Budgeting & Finance / Winter 2012

 TABLE 3
Dependent Variable: Symmetrical Percent Error
Omit NYCOMB Omit FCB Omit IBO Omit NYCC Omit DCNYC
Model 1a Model 1b Model 1c Model 1d Model 1e

Coef. Std. err. Coef. Std. err. Coef. Std. err. Coef. Std. err. Coef. Std. err.
Out Year −0.039 0.004b −0.039 0.004b −0.039 0.004b −0.039 0.004b −0.039 0.004b
Second −0.017 0.016 −0.017 0.016 −0.017 0.016 −0.017 0.016 −0.017 0.016
NYCC 0.002 0.012 0.016 0.018 −0.014 0.014 0.014 0.010
IBO 0.016 0.012c 0.030 0.024 0.014 0.014 0.028 0.014a
DCNYC −0.012 0.013 0.002 0.017 −0.028 0.014a −0.014 0.010
FCB −0.014 0.024 −0.030 0.024 −0.016 0.018 −0.002 0.017
NYCOMB 0.014 0.024 −0.016 0.012c −0.002 0.012 0.012 0.013
R-squared 0.723 0.723 0.723 0.723 0.723
Adj. R-sq. 0.680 0.680 0.680 0.680 0.680
F 67.9 (84,540) 67.9 (84,540) 67.9 (84,540) 67.9 (84,540) 67.9 (84,540)
p-value (F) 0.000 0.000 0.000 0.000 0.000
Note: Control (interaction) variables not shown.
All models controlled for expenditure year and budget year fixed effects.
Models 1a through 1e sequentially include one forecaster in the base model to test for difference with the others.
Significance: a 10%, two tail; b 1% one tail; c 10%, one tail.

distinguished from no bias at the intercept of the regression model (i.e., in the budget year).
The answer is that only the FCB can be distinguished from zero.
We are also interested in whether the forecasters can be distinguished from each other.
For that question we turn to Table 3, which shows the effect when each of the forecaster
dummies is omitted from the model sequentially. This exclusion puts the excluded forecaster
into the base model and tests for the hypothesis that this forecaster is significantly different
from the other forecasters. Where IBO or NYCC is compared with NYCOMB or FCB,
directional hypotheses are appropriate (IBO or NYCC should have positive coefficients,
NYCOMB or FCB should have negative coefficients), otherwise nondirectional hypotheses
are required. IBO forecast bias is statistically different with a positive coefficient from
NYCOMB, p = 0.1, and from DCNYC, p = 0.1.
Hypothesis 2 is that there is a revenue category effect that is uncertain for property tax,
negative (underforecasting) for other New York City own-source revenue, and uncertain for
federal and state categorical grants. Model 2 shows underforecasting (negative coefficients)
for all other taxes, miscellaneous and other city revenue, and federal categorical grants. It
shows positive (over forecasting) for property tax and state categorical grants. Property tax,
all other taxes, and Federal Categorical Revenue are statistically significant. An important
potential limitation is that because of the way some values were imputed, it is likely that
the variance in property tax is understated, so the level of significance may be biased; no
known data source is available to examine this concern.

Williams / The Politics of Forecast Bias 13

 The finding with respect to property tax is more consistent with the view that it represents
demand for services reflecting the city council’s partial control over property tax, than with
its representing technical simplicity, which should lead to essentially no bias. This finding
suggests a need for more exploration of these factors as they contribute to forecast accuracy
or bias. The finding with respect to federal categorical grants suggests more concern over
expenditure impact than any impact on grant seeking.
By multiplying the statistically significant coefficients from Model 2 by the proportions
in Table 1 the net bias is an approximately 1.6 percent underforecast in the budget year.
Hypothesis 3 is that there is a negative effect, that is, compound underforecasting bias,
for forecast horizon. The out year variable13 is shown in all models and has a coefficient
ranging from −.039 to −.040 (−3.9 to −4.0 percent) with significance at the 1 percent
level in all cases. This finding is of particular interest because it is consistent with the view
that New York City’s forecast bias is substantially larger (approximately 4 percent larger)
in each subsequent year following the budget year.14 This bias is added to the first year
bias of 1.6 percent. By the third subsequent year there is an approximately 13.6 percent
underforecasting bias.15 The implication is that during the period of this analysis there is
substantial reason to suspect that New York City’s structural budget deficit is overstated,
although a rigorous analysis of the expenditure forecast would be required to fully determine
this matter. Hypothesis 4 is that there is a month of origin effect that is negative for May,
uncertain for June, and positive for July; and Hypothesis 5 is that there is a positive effect
for the second forecast in the same cycle. The coefficient for May in Model 3 is consistent
with this hypothesis; however, it is not significant. Coefficients for June and July are not
significant and the July coefficient has an unexpected sign. All four models include the
variable Second Forecast. The coefficient is not stable and not significant.

DISCUSSION AND CONCLUSION

This paper has examined the impact of forecasters, horizons, revenue categories, and forecast
timing in relation to decision making on forecast bias or accuracy. For the most part
forecasters tend to report forecasts that agree rather than compete. This finding suggests
that five separate forecasts are excessive. However, it may be unwise to reduce the forecasts
to one. The evidence here is not sufficient to show that the convergence is toward NYCOMB
rather than toward some other value that is less biased than NYCOMB would be without

13. Out year is the only continuous variable in the model.

14. A potential alternate dependent variable is percent error, which was examined and is shown in the
expression: F −A
A
× 100, with this dependent variable The out year coefficient is = −0.033 (−3.3 percent), so
the four year bias would be −1.6, −4.9, −8.2, −11.5 percent, the compound value (see next footnote) is 12
percent.
15. This amount is an additive, rather than compounded, value. The compound value is (1.016 × 1.0393 )
− 1 = 0.1396, or 14 percent; however, this amount could be slightly overstated because of the original use of
the symmetrical formula.

14 Public Budgeting & Finance / Winter 2012

any competition. Based on Model 1c the highest level of competition is between IBO and
DCNYC, with OMB as a mandatory forecast because of charter provisions. The other two
can be dispensed with, particularly FCB, for which there is some evidence of bias. NYCC
might retain its capacity and put it to better use by conducting the sort of evaluation studies
reported in this paper. Additional research on this topic could examine NYCOMB forecast
over a longer period of time and the effect of introduction of the IBO competitor at the
beginning of the 2000s.
Forecast bias (underforecasting) increases over longer horizons; consequently claims of
structural budget deficit may be suspect. The approximate value of the revenue forecast
bias in the second and third out years, that is, in the time that would relate to the claim
of structural deficit, is approximately −10 and −14 percent, respectively. While there is a
perception of “structural deficit,” available evidence is inconclusive and suggests a need for
rigorous examination of all New York City forecasting practice. Although there may be
short-term imbalances during the aftermath of the recent severe recession, that is not the
same as a structural deficit. Policy making aimed at correcting a structural deficit could
be based on significantly biased forecasts. In recent times New York City has generated
large surpluses in several years, even at the beginning of a severe recession. Future research
should investigate the accuracy of expenditure forecasting and the allocation of surpluses.
There is an over forecasting bias of 13.3 percent with property tax, which suggests that
political control over property tax by the city council may affect the direction and size of
forecasting bias. This finding requires additional examination.
There is an underforecasting bias in two substantial revenue categories. The bias in federal
categorical grants suggests more concern over expenditure impact than use of expectations
in grant seeking. For all other taxes, there is a statistically significant underforecasting bias
of 11.2 percent. This bias in the two larger categories (all other taxes and miscellaneous and
other city revenue) is consistent with the view that New York City prefers to understate its
revenue when making its budget. Further research may look into explanation of this bias
and examine when it may have arisen over the history of this forecast.
Forecasts reported in June have an underforecasting bias. This result is intriguing, but
requires further study before it can be interpreted.
The study suggests further examination of the bias associated with revenue categories,
time within the budget cycle, and forecast horizon. Expenditure forecast accuracy should be
studied to determine the net total accuracy of structural deficit forecasting. By examining a
single forecaster over a longer period of time, revenue category effect can be more thoroughly
examined. In recent times, New York City has published its forecasts each month, so it is
possible to extend the examination of forecast timing.

REFERENCES

Armstrong, J. Scott. 2006. “Findings from Evidence-Based Forecasting: Methods for Reducing Forecast Error.” International Journal
of Forecasting. 22 (3): 583–598. doi: 10.1016/j.ijforecast.2006.04.006.

Williams / The Politics of Forecast Bias 15

Batchelor, Roy. 2007. “Bias in Macroeconomic Forecasts.” International Journal of Forecasting. 23 (2): 189–203. doi:
10.1016/j.ijforecast.2007.01.004.
Bland, Robert L. 2007. A Budgeting Guide for Local Government. Second Edition. Washington, D.C.: ICMA Press.
Bretschneider, Stuart, Jeffrey J. Straussman, and Daniel Mullins. 1988. “Do Revenue Forecasts Influence Budget Setting? A Small
Group Experiment.” Policy Sciences. 21 (4): 305–325.
Burkhead, Jesse. 1956. Government Budgeting. New York: Wiley.
Butler, Kirt C. and Larry H. P. Lang. 1991. “The Forecast Accuracy of Individual Analysts: Evidence of Systematic Optimism and
Pessimism.” Journal of Accounting Research. 29 (1): 150–156.
Chen, David W. and Michael Barbaro. 2008. “Bloomberg’s Stewardship Is Mixed, Fiscal Experts Say.” The New York Times, October
2, 2008, Section B; Column 0; Metropolitan Desk; Pg. 1.
Choate, G. Marc and Fred Thompson. 1988. “Budget Makers as Agents: A Preliminary Investigation of Discretionary Behavior
under State-Contingent Rewards.” Public Choice. 58 (1): 3–20.
———. 1990. “Biased Budget Forecasts.” Journal of Economic Behavior and Organization. 14 (3): 425–434.
Clement, Michael B. and Senyo Y. Tse. 2005. “Financial Analyst Characteristics and Herding Behavior in Forecasting.” Journal of
Finance. 60 (1): 307–341.
Cooper, Michael. 2004. “A Proposal for Few Cuts, No Layoffs and Even a Tax Rebate.” The New York Times, January 16, 2004,
Section B; Column 1; Metropolitan Desk; Pg. 6.
Crain, W. Mark and Timothy J. Muris. 1995. “Legislative Organization of Fiscal Policy.” Journal of Law and Economics. 38 (2):
311–333.
Dekker, Mark, Karel van Donselaar, and Pim Ouwehand. 2004. “How to Use Aggregation and Combined Forecasting to Improve
Seasonal Demand Forecasts.” International Journal of Production Economics. 90 (2): 151–167. doi: 10.1016/j.ijpe.2004.02.004.
Fildes, Robert, Michèle Hibon, Spyros Makridakis, and Nigel Meade. 1998. “Generalising About Univariate Forecasting Methods:
Further Empirical Evidence.” International Journal of Forecasting. 14 (3): 339–358. doi: 10.1016/s0169-2070(98)00009-0.
Gardner, Everette S., Jr. and Ed McKenzie. 1985. “Forecasting Trends in Time Series.” Management Science. 31 (10): 1237–1246.
Garman, Christopher, Stephan Haggard, and Eliza Willis. 2001. “Fiscal Decentralization: A Political Theory with Latin American
Cases.” World Politics. 53 (2): 205–236.
Heller, William B. 1997. “Bicameralism and Budget Deficits: The Effect of Parliamentary Structure on Government Spending.”
Legislative Studies Quarterly. 22 (4): 485–516.
Krause, George A. and James W. Douglas. 2006. “Does Agency Competition Improve the Quality of Policy Analysis? Evidence from
OMB and CBO Fiscal Projections.” Journal of Policy Analysis and Management. 25 (1): 53–74.
Laster, David, Paul Bennett, and In Sun Geoum. 1999. “Rational Bias in Macroeconomic Forecasts.” Quarterly Journal of Economics.
114 (1): 293–318.
Lawrence, Michael, Marcus O’Connor, and Bob Edmundson. 2000. “A Field Study of Sales Forecasting Accuracy and Processes.”
European Journal of Operational Research. 122 (1): 151–160. doi: 10.1016/s0377-2217(99)00085-5.
Levy, Clifford J. 1996. “New York City Hears of Surplus, but Big Budget Gaps Lie Ahead.” The New York Times, December 11, 1996.
MacManus, Susan A. and William J. Jr. Pammer. 1990. “Cutbacks in the Country: Retrenchment in Rural Villages, Townships, and
Counties.” Public Administration Quarterly. 14 (3): 302–323.
Makridakis, S., A. Andersen, R. Carbone, R. Fildes, M. Hibon, R. Lewandowski, J. Newton, E. Parzen, and R. Winkler. 1982.
“The Accuracy of Extrapolation (Time Series) Methods: Results of a Forecasting Competition.” Journal of Forecasting. 1 (2):
111–153.
Makridakis, S., C. Chatfield, M. Hibon, M. Lawrence, T. Mills, K. Ord, and L.R.F. Simmons. 1993. “The M2-Competition: A
Real-Time Judgmentally Based Forecasting Study.” International Journal of Forecasting. 9 (1): 5–22.
Makridakis, S. and M. Hibon. 2000. “The M3-Competition: Results, Conclusions and Implications.” International Journal of Fore-
casting. 16 (4): 451–476.
Makridakis, Spyros, Michele Hibon, and Claus Moser. 1979. “Accuracy of Forecasting: An Empirical Investigation.” Journal of the
Royal Statistical Society. Series A (General). 142 (2): 97–145.
Olsen, Robert A. 1996. “Implications of Herding Behavior for Earnings Estimation, Risk Assessment, and Stock Returns.” Financial
Analysts Journal. 52 (4): 37–41.
Paleologou, S. 2005. “Political Manoeuvrings as Sources of Measurement Errors in Forecasts.” Journal of Forecasting. 24 (5): 311–324.
Pressman, Steven. 2004. “What Is Wrong with Public Choice.” Journal of Post Keynesian Economics. 27 (1): 3–18.

16 Public Budgeting & Finance / Winter 2012

Rodgers, Robert and Philip G. Joyce. 1996. “The Effect of Underforecasting on the Accuracy of Revenue Forecasts by State
Governments.” Public Administration Review. 56 (1): 48–56.
Rohatyn, Felix G. 1994. “A Leaner New York.” The New York Times, December 18, 1994.
Rose, Richard. 1985. “Maximizing Tax Revenue While Minimizing Political Costs.” Journal of Public Policy. 5 (3): 289–320.
Rubin, Irene S. 1987. “Estimated and Actual Urban Revenues: Exploring the Gap.” Public Budgeting and Finance. 7 (4):83–94.
Shefter, Martin. 1992. Political Crisis/Fiscal Crisis: The Collapse and Revival of New York City. New York: Columbia University
Press.
Smith, Stanley K. and Terry Sincich. 1991. “An Empirical Analysis of the Effect of Length of Forecast Horizon on Population
Forecast Errors.” Demography. 28 (2): 261–274.
Voorhees, William R. 2006. “Consistent Underestimation Bias, the Asymmetrical Loss Function, and Homogenous Sources of Bias
in State Revenue Forecasts.” Journal of Public Budgeting, Accounting and Financial Management. 18 (1): 61–76.
Weaver, R. Kent. 1986. “The Politics of Blame Avoidance.” Journal of Public Policy. 6 (4): 371–398.

APPENDIX ON VARIABLES
Dependent Variables

SPE_RCD = symmetrical percent error as defined in the main text. The sublabel RCD
reflects combining miscellaneous and other revenue to exclude negative revenue.
PE_RCD = percent error as defined in the main text.

Independent Variables

Out Year = years past the first year of the forecast coded 0 for the upcoming budget year
and 1, 2, and 3 for the three subsequent years.
Second = coded 1 if the record is for the Second forecast made by the forecaster during the
same cycle, otherwise 0.

Forecaster

NYCC = coded 1 if the forecaster is the New York City Comptroller, otherwise 0.
IBO = coded 1 if the forecaster is the Independent Budget Office, otherwise 0.
DCNYC = coded 1 if the forecaster is the State Deputy Comptroller for New York City,
otherwise 0.
FCB = coded 1 if the forecaster is the Financial Control Board, otherwise 0.
NYCOMB = coded 1 if the forecaster is the New York City Office of Management and
Budget, otherwise 0.

Revenue Category

Property = coded 1 if the revenue category is property tax, otherwise 0.

AllOtherTax = coded 1 if the revenue category all other taxes, otherwise 0.

Williams / The Politics of Forecast Bias 17

Miscellaneous = coded 1 if the revenue category is miscellaneous and other, otherwise 0.
FedCatGrants = coded 1 if the revenue category is federal categorical grants, otherwise 0.
StateCatGrants = coded 1 if the revenue category is state categorical grants, otherwise 0.

Month

May = coded 1 if the forecast origin is in May, otherwise 0.

June = coded 1 if the forecast origin is in June, otherwise 0.
July = coded 1 if the forecast origin is in July, otherwise 0.

Control Independent Variables

FCY03 through FCY07 = forecast origin year 2003 through 2007, individually, coded 1 for
the respective year, otherwise 0.
BY04 through BY09 = budget year (forecast target year) 2004 through 2009, coded 1 for
the respective year, otherwise 0.

Interaction Variables

Four-way interaction variables, which involves multiplying Category, Forecaster, Month,

and forecast origin year indicator variables. Three-way interaction variables were created
using various combinations of these excluding variables of interest.

18 Public Budgeting & Finance / Winter 2012