Evaluation Plan for Visit Alexandria

April 2015
Prepared by:
PAPA 6224
Martha Ascencio
Kulsoom Basharat
Sherry Saurage
TiArre Stromer
Virginia Tech Center for Public Administration and Policy (CPAP)

City of Alexandria
Alexandria, Virginia

Acknowledgement:
The CPAP team would be negligent without recognizing the hard work that Visit Alexandria staff
members have put forth in creating the Visit Alexandria performance measures.
Special thanks are due to the Visit Alexandria organization which has provided us with this
professional development opportunity.
Finally, we would like to sincerely thank our professor, Dr. Adam Eckerd, for his conscientious
guidance, insight, and support throughout the project.

Evaluation Plan for Visit Alexandria

EXECUTIVE SUMMARY
According to Clemons and McBeth in Public Policy Praxis: A Case Approach for Understanding
Policy and Analysis, once a policy has been adopted, it should be evaluated for effectiveness in
meeting its goals (Clemons & McBeth, 2009, p. 134). In our effort to learn about design,
implementation, and evaluation of public policy, we were introduced to Visit Alexandria, a local
501 (c) (6) nonprofit organization charged with increasing visitor spending and tax revenue
through: more people, staying longer, and spending more (Fiore, 2015). Visit Alexandria is
independent of Alexandria City government and receives a significant level of funding from
general tax revenues through a contractual relationship. As a group learning project, we were
charged to critique Visit Alexandrias current evaluation practices and indicate areas of
opportunity that are not currently being met and offer areas in need of further development.
Additionally, we reviewed publicly accessible Visit Alexandria data, researched similar
organizations (such as data from the Williamsburg Visitors Center (Williamsburg, 2015)), and
conducted internal brainstorming sessions to frame the development of a comprehensive
evaluation plan in an effort to account for the goals of the organization. This research led to the
methodology allowing for a high-level comparison of current practices to what we find as
valuable areas to critique. We recognize the limitations of our analysis is due to the lack of
experience in public policy evaluation processes; however, we reflected on various in-class
discussions, research, course materials, and personal experiences in order to formulate our
recommendations that would assist Visit Alexandria and the City to meet its overarching goal of
increasing economic benefits of tourism for Alexandria.

Extraordinary Alexandria

Page 2 of 17

Evaluation Plan for Visit Alexandria

Table of Contents
EXECUTIVE SUMMARY .......................................................................................................... 2
1
1.1
2
2.1
3
3.1

INTRODUCTION................................................................................................................. 4
Evaluation Importance and Process................................................................................. 4
PART I- ASSESSMENT OF AS-IS STATE ...................................................................... 5
Analysis of Current Performance Measures ................................................................... 5
PART II- EVALUATION PLAN ........................................................................................ 5
Approach ............................................................................................................................ 5

3.1.2 Performance Measures (Goals, Indicators, Outcomes) .................................................... 6

4

DETAILED EVALUATION PLAN .................................................................................. 10

HIGH LEVEL LOGIC MODEL ....................................................................................... 15

CONCLUSION ................................................................................................................... 16

REFRENCES ...................................................................................................................... 17

Extraordinary Alexandria

Page 3 of 17

Evaluation Plan for Visit Alexandria

1 INTRODUCTION
The City of Alexandria understands the importance and value of tourism. It appreciates the
increase of local tax revenue, creating jobs, strengthening local businesses and building
Alexandrias image and community pride. With these goals in mind, the City of Alexandria has
partnered with Visit Alexandria, a non-profit organization that promotes the valuable
attractions of the City and aims to increase visitor spending and tax revenue through the
mindset, more people, staying longer and spending more. With this goal in mind, the CPAP
Team proceeded to evaluate the organization.
1.1
Evaluation Importance and Process
Evaluation is important to Visit Alexandria because it needs to monitor progress towards
achieving its goals and to maintain accountability to its partnership with the City of Alexandria.
Effective measures can give Visit Alexandria the confirmation it needs to illustrate
program/project performance, assign accountability, and provide effective stewardship to its
partners, residents, and visitors.
Keeping Visit Alexandrias mission and priorities in mind, the methodology the CPAP team
used to develop the evaluation plan was to first review the current performance measures to
decipher how effective the goals and indicators were in measuring the overarching goal for the
organization. Then, performance measures for other organizations (similar to Alexandria) were
examined and through a combination of internal brainstorming sessions and research, a
critique of the current goals and indicators were developed leading to the following
recommendations on goal setting and assessing performance to meet the needs of increasing
the economic benefits of tourism in Alexandria.
Extraordinary Alexandria

Page 4 of 17

Evaluation Plan for Visit Alexandria

2 PART I- ASSESSMENT OF AS-IS STATE

2.1
Analysis of Current Performance Measures
The long term objective that Visit Alexandria has specified is, Increase the economic
benefits of tourism to the City by setting the goal of increasing visitor spending and tax
revenue through more people, staying longer, and spending more. In addition, Visit
Alexandria has also identified multiple indicators, or measures, such as number of visitors to
Alexandria, dollars of hotel revenues, and number of social media followers and Visit Alexandria
members.
While these are all important measures that provide valuable information to Visit
Alexandria, and technically there are no invalid measures, much of the data being captured and
reported represent outputs, rather than outcomes, and some of the goals are vague and do not
encompass the key mission of the organization. Therefore, a portion of the current measures
are not providing the organization with meaningful information on how well it is performing
towards its mission.
It is not necessarily the specific measurements that are of importance, but the value in
the progress towards the results and the extent to which the outputs affect the outcomes. It is
recognized that Visit Alexandria must continue to provide specific metrics that the City has
requested, however, perhaps a case could be presented in the future that is in support of
introducing new or a modified metrics that better represent Visit Alexandrias performance and
accomplishments.

3 PART II- EVALUATION PLAN

3.1

Approach

Extraordinary Alexandria

Page 5 of 17

Evaluation Plan for Visit Alexandria

The approach taken to develop the evaluation design and measurement framework is
based on the use of a logic model (Kellogg, 2006). The model is comprised of five core
components which include, inputs, activities, outputs, outcomes, and impacts which
demonstrate how actions and investments lead to results. The framework was tailored to
address the specific objectives identified by Visit Alexandria while incorporating the collection
and analysis of information about its target activities and characteristics through existing and
accessible data with the ultimate purpose of being able to make judgments about the
organization, to improve its effectiveness, and/or to inform decisions.
The CPAP team analyzed the scholarly research and practices to identify a number of
actionable recommendations for designing, evaluating, and measuring Visit Alexandrias goals.
The logic model and a performance measures framework were used to develop the
recommended indicators and goals. The focus was on the effectiveness measurement using
the logic model. We reviewed the current indicators/outputs to see how effectively they
measured the type and number of data being produced by the process and then we developed
new goals that leveraged the old/modified outputs/indicators that lead to meaningful
outcomes.
3.1.2 Performance Measures (Goals, Indicators, Outcomes)
Each goal and outcome has been tied to Visit Alexandrias long term objective which is to
increase the economic benefits of tourism to the city.
Current Goal #1: People Visit Alexandria/New Goal #1: Better understand the fluctuation in
data among the members and communicate results
The concern with this goal was that it currently appears to be an output rather than a goal.
People Visiting Alexandria is generic and covers a wide expanse and the goal was questioned
Extraordinary Alexandria

Page 6 of 17

Evaluation Plan for Visit Alexandria

whether anyone can just visit the City without spending money, so it was determined that this
goal did not accurately match up to the indicators nor did it specifically link back to the
overarching goal. Therefore, the recommendation to revise Goal #1 is to use the current
indicators so that Visit Alexandria can better understand the fluctuation in data among the
members and communicate results. This goal ideally will involve the identification of consumer
spending trends and gaps in performance from the use of existing data and should involve
members (business owners) to address any shortfalls identified. Through the trend analysis, the
anticipated outcome of this goal is to identify opportunities and resolve any shortfalls in order
to enhance visitor experience (see Table 1 Goal 1).
Current Goal #2: Market Alexandria as a leisure visitor destination/New Goal #2: Assess
patterns of behavior and Increase awareness of leisure and recreational opportunities in
Alexandria
The issue with this goal is that it is very broad and does not provide tangible direction to
achieve the overarching goal and is written as an output. The goal has been revised by breaking
it into two parts- first patterns in consumer behavior should be assessed. The next goal would
be to increase the awareness of leisure and recreational activities that are offered within the
City. The current indicators listed are sufficient and will provide the measures for the new goals
to be assessed, however the indicator Number of visitors to the visitor center was deleted
and replaced with the Number of inquiries made at the visitor center to be more specific in
capturing the objective. The outcome of Goal #2 is to make Alexandria a premier tourist
destination (see Table 2 Goal 2).

Extraordinary Alexandria

Page 7 of 17

Evaluation Plan for Visit Alexandria

Current Goal #3: Market Alexandria as a business meeting destination /New Goal #3: Increase
the number of businesses that take advantage of facilities in Alexandria and Increase the
awareness of business gathering opportunities in Alexandria
Similar to Goal#2, Goal#3 is very broad and does not provide tangible direction in achieving the
overarching goal. The outcome is to enhance the business visitor experience and the City can
achieve that by increasing the number of businesses that take advantage of facilities in
Alexandria and by spreading awareness on business gathering opportunities available in the
City. While the existing measures in place support the goal and outcome, three additional
measures were developed to further capture business visitor sentiments as well as supplement
the financial relationships already being analyzed from available data (see Table 3 Goal 3).
Current Goal #4/5: Develop content for earned media stories in print, online, or broadcast
and Cultivate digital and traditional media to promote Alexandria as a visitor
destination/New Goal #4 (Combined 4&5): Develop brand awareness and Increase the
intention to visit website (Visit Alexandria page, blog, social media)
These two goals were combined and revised to clearly pave the path towards the expected
outcome rather than just defining its outputs. Two additional measures were introduced as well
which will provide useful information to Visit Alexandria on how well it is performing in
spreading its message across the various forms of media available (see Table 4 Goal 4).
Current Goal #6: Work with Visit Alexandria members (local businesses) to coordinate
activities to increase marketing effectiveness to promote Alexandria as a visitor
destination/New Goal #5 Increase the Number of members, Assess the perception of
membership advantages, and Increase collaboration and engagement with members

Extraordinary Alexandria

Page 8 of 17

Evaluation Plan for Visit Alexandria

Even though this goal is valuable it does not state the purpose/objective of working with Visit
Alexandria members. By revising the goal and including a few additional evaluation measures,
Visit Alexandria can be equipped to better understand its membership base and position itself
to nurture its relationships and gain unique perspectives from the field. Additionally, the path
to achieving the outcome of enhancing member experience now has greater visibility while
directly supporting the organizational objective of Visit Alexandria throughout the community
at large (see Table 5 Goal 5).

Extraordinary Alexandria

Page 9 of 17

Evaluation Plan for Visit Alexandria

4 DETAILED EVALUATION PLAN

Goal

Comments

Current Goals

Current Indicators

New Goals

New Outcomes

Increase the Economic Benefits of Tourism to the City

This goal needs to be further
defined. The current goal is
an output.

Goal #1

This goal ideally should

involve the identification of
consumer spending trends
and performance gaps and
involve members (business
owners) to address any
shortfalls.
The current indicators will
help with the analysis of
spending.

People visit
Alexandria

Number of visitors to
Alexandria (millions)
Dollars of total visitor spending
(millions)

Better understand the

fluctuation in data among
the members and
communicate results.

Dollars of hotel revenues

(millions)
Dollars of rooms tax dollars
collected by City (millions)
Dollars of restaurant revenues
(millions)
Dollars of meals tax dollars
collected by City (millions)
Dollars of retail revenues
(billions)
Dollars of sales tax collections
by City (millions)
Table 1 Goal 1

Extraordinary Alexandria

Page 10 of 17

Enhance the visitor experience.

Evaluation Plan for Visit Alexandria

The current goal is very

broad and does not provide
tangible direction to achieve
the overarching goal. The
current indicators will
provide measures for the
new goals.

Market Alexandria
as a leisure visitor
destination

Goal #2

Percent of survey respondents

stating that the advertising
campaign is effective or very
effective in portraying
Alexandria as a place they
would enjoy visiting for leisure
Percent of visitors likely to
recommend Alexandria to
family and friends

Assess patterns of
behavior.

Percent of survey respondents

recalling they have seen or
heard advertising for travel to
Alexandria (unaided)

Increase awareness of
leisure and recreational
opportunities in
Alexandria.

Ratio of advertising return on

investment for visitor (related
to leisure) expenditures
Ratio of advertising return on
investment for tax receipts
(related to leisure) to the City's
general fund
Number of inquiries made at
the Visitor center.
Number of visitors to the
Visitor Center

Table 2 Goal 2

Extraordinary Alexandria

Page 11 of 17

Make Alexandria a premier

tourist destination.

Evaluation Plan for Visit Alexandria

The current goal is very

broad and does not provide
tangible direction in
achieving the overarching
goal.
The current indicators will
provide measures for the
new goals.

Market Alexandria
as a business
meeting
destination

Ratio of advertising return on

investment for tax receipts
(related to business) to the
City's general fund

Increase the number of

businesses that take
advantage of facilities in
Alexandria.

Ratio of advertising return on

investment for visitor (related
to business) expenditures

Goal #3

Dollars in actualized sales

revenue from leads generated
by Visit Alexandria activities
(millions)
Dollars in projected revenue
from sales leads generated by
Visit Alexandria activities
(millions)
Dollars of economic impact
from business meetings
(millions)
Number of direct sales
meetings with groups and
meeting planners
Percent of survey respondents
recalling they have seen or
heard advertising for business
meetings to Alexandria
(unaided)

Increase the awareness

of business gathering
opportunities in
Alexandria.

Table 3 Goal 3

Extraordinary Alexandria

Page 12 of 17

Enhance the business visitor

experience.

Evaluation Plan for Visit Alexandria

These two goals were

combined and revised to
clearly outline the path to
the expected outcome.

Goal #4

Develop content
for earned media
stories in print,
online, or
broadcast

Number of earned media

stories in print, online, or
broadcast

Cultivate digital
and traditional
media to promote
Alexandria as a
visitor destination

Number of unique web site

visitors (millions)

Develop brand awareness Cultivate digital and traditional

(e.g., Extraordinary
media to promote Alexandria as a
Alexandria).
visitor destination.

Number of international earned

media stories in print, online, or
broadcast
Increase the intention to
visit website (Visit
Alexandria page, blog,
social media).

Number of new web site

visitors (millions)
Duration of visit on the website

Number of social media

followers
number of retweets (Twitter),
likes/shares/repost (Facebook
and Instagram)
Table 4 Goal 4

Extraordinary Alexandria

Page 13 of 17

Evaluation Plan for Visit Alexandria

This goal is valuable;

however it does not state
the purpose/ objective of
working with Visit
Alexandria members.

Goal #5

Work with Visit

Alexandria
members (local
businesses) to
coordinate
activities to
increase marketing
effectiveness to
promote
Alexandria as a
visitor destination

Number of Visit Alexandria

members

Increase the number of

members.

Percent of members
participating in Visit Alexandria
programs
Number of successful
networking engagements
facilitated by Visit Alexandria.
(this would also be assessed
through survey)
Number of attendees at
research marketing forums who
use local tourism-related
businesses use marketing data
Percent of renewal rate of Visit
Alexandria members

Assess the perception of

membership advantages.

Number of satisfied members

(conduct survey)/ Collecting
feedback from members
Number of members receptive
to feedback

Increase collaboration
and engagement with
members.

Table 5 Goal 5

Extraordinary Alexandria

Page 14 of 17

Enhance the member experience.

Evaluation Plan for Visit Alexandria

5 HIGH LEVEL LOGIC MODEL

Outputs

Inputs

Outcomes -- Impact

Activities

Short

Long

Activities
-Advertising and Marketing
-Education
-Material and
Resources used by Visit
Alexandria to serve its
clients

-Enhance the visitor

experience.

-Website
-Social Media Sites
-Cultivate digital and

traditional media to
promote Alexandria as a
visitor destination.

-Equipment
-Staff
-Money
-Facilities

Outputs (unit of service)

-Number of Visitors to Alexandria

-Enhance the business

visitor experience.
-Enhance the member
experience.

-Number of Visit Alexandria members

-Dollars of Total Visitor Spending
-Dollars of Retail Revenues
-Number of Inquiries Made at Visitor Center
Table 6 Logic Model

Extraordinary Alexandria

-Make Alexandria a
premier tourist
destination.

Page 15 of 17

Evaluation Plan for Visit Alexandria

6 CONCLUSION
Visit Alexandria will need to assess how effective its goals are and how successful they are
in reaching the desired outcomes on an annual basis. In the future, Visit Alexandria may
consider measuring resident and visitor satisfaction, or increases in the collaboration/support
of local businesses and the City Council, so that it can gain insight into how the activities led by
the organization are performing to meet the needs of, and what impact they have on, the
community in general.
When developing future measures, Visit Alexandria may also want to consider incorporating
measures that are important to the citizens, city administrators, and key stakeholders, as well
as measures that other similar jurisdictions have found to be successful in practice through
Town Hall meetings or conducting key stakeholder surveys. To aid the process, Visit Alexandria
may benefit from constructing a logic model similar to the framework that was utilized during
the development of this proposed evaluation plan.
It is further recommended that Visit Alexandria continue to measure the specific indicators
that are requested by the City, but to also strive to incorporate some additional measures that
will aid in Visit Alexandria better understanding the genuine impact(s) it is making in the
community.

Page 16 of 17

Evaluation Plan for Visit Alexandria

7 REFRENCES
City of Fredericksburg. March 2012. Tourism Development Plan.
Clemons, R. and McBeth, M. 2009. Public Policy Praxis: A Case Approach for Understanding
Policy and Analysis (2nd Edition). Pearson: New York.
Frechtling, D. 2006. An Assessment of Visitor Expenditure Methods and Models. Journal of
Travel Research.
Virginia Tourism Corporation. Tourism Works for Virginia.
http://www.vatc.org/research/travelerattitudes/ on April 4, 2015.

Retrieved

from

W.K. Kellogg Foundation. Feb 2006. W.K. Kellogg Foundation Logic Model Development Guide.
Clemons, R., & McBeth, M. (2009). Public Policy Praxis: A Case Approach for Understanding
Policy and Analysis. New York: Pearson.
Fiore, V. (2015, March). Office Manager and Research Analyst. (C. A. Team, Interviewer)
Williamsburg. (2015, April). Retrieved from Visit Williamsburg: www.visitwilliamsburg.com

Page 17 of 17