Research II / 1

Analyzing and Interpreting Information

Analyzing quantitative and qualitative data is often the topic of advanced research and evaluation
methods courses. However, there are certain basics which can help to make sense of reams of data.

Always start with your research goals

When analyzing data (whether from questionnaires, interviews, focus groups, or whatever),
always start from review of your research goals, i.e., the reason you undertook the research in the first
place. This will help you organize your data and focus your analysis. For example, if you wanted to
improve a program by identifying its strengths and weaknesses, you can organize data into program
strengths, weaknesses and suggestions to improve the program. If you wanted to fully understand how
your program works, you could organize data in the chronological order in which customers or clients go
through your program. If you are conducting a performance improvement study, you can categorize data
according to each measure associated with each overall performance result, e.g., employee learning,
productivity and results.

Basic analysis of "quantitative" information

(for information other than commentary, e.g., ratings, rankings, yes's, no's, etc.):

1. Make copies of your data and store the master copy away. Use the copy for making edits, cutting
and pasting, etc.
2. Tabulate the information, i.e., add up the number of ratings, rankings, yes's, no's for each
question.
3. For ratings and rankings, consider computing a mean, or average, for each question. For example,
"For question #1, the average ranking was 2.4". This is more meaningful than indicating, e.g.,
how many respondents ranked 1, 2, or 3.
4. Consider conveying the range of answers, e.g., 20 people ranked "1", 30 ranked "2", and 20
people ranked "3".

Basic analysis of "qualitative" information

(respondents' verbal answers in interviews, focus groups, or written commentary on questionnaires):

1. Read through all the data.

2. Organize comments into similar categories, e.g., concerns, suggestions, strengths, weaknesses,
similar experiences, program inputs, recommendations, outputs, outcome indicators, etc.
3. Label the categories or themes, e.g., concerns, suggestions, etc.
4. Attempt to identify patterns, or associations and causal relationships in the themes, e.g., all people
who attended programs in the evening had similar concerns, most people came from the same
geographic area, most people were in the same salary range, what processes or events respondents
experience during the program, etc.
5. Keep all commentary for several years after completion in case needed for future reference.

Interpreting information

Attempt to put the information in perspective, e.g., compare results to what you expected,
promised results; management or program staff; any common standards for your products or services;
original goals (especially if you're conducting a program evaluation); indications or measures of
accomplishing outcomes or results (especially if you're conducting an outcomes or performance
evaluation); description of the program's experiences, strengths, weaknesses, etc. (especially if you're
conducting a process evaluation).

Consider recommendations to help employees improve the program, product or service;

conclusions about program operations or meeting goals, etc.
 Research II / 2

Record conclusions and recommendations in a report, and associate interpretations to justify your
conclusions or recommendations.

Reporting Results

The level and scope of content depends on to whom the report is intended, e.g., to funders /
bankers, employees, clients, customers, the public, etc.

Be sure employees have a chance to carefully review and discuss the report. Translate
recommendations to action plans, including who is going to do what about the research results and by
when.

Funders / bankers will likely require a report that includes an executive summary (this is a
summary of conclusions and recommendations, not a listing of what sections of information are in the
report -- that's a table of contents); description of the organization and the program, product, service, etc.,
under evaluation; explanation of the research goals, methods, and analysis procedures; listing of
conclusions and recommendations; and any relevant attachments, e.g., inclusion of research
questionnaires, interview guides, etc. The funder may want the report to be delivered as a presentation,
accompanied by an overview of the report. Or, the funder may want to review the report alone.

Be sure to record the research plans and activities in a research plan which can be referenced
when a similar research effort is needed in the future.

Who Should Carry Out the Research?

Ideally, the organization's management decides what the research goals should be. Then a
research expert helps the organization to determine what the research methods should be, and how the
resulting data will be analyzed and reported back to the organization.

If an organization can afford any outside help at all, it should be for identifying the appropriate
research methods and how the data can be collected. The organization might find a less expensive
resource to apply the methods, e.g., conduct interviews, send out and analyze results of questionnaires,
etc.

If no outside help can be obtained, the organization can still learn a great deal by applying the
methods and analyzing results themselves. However, there is a strong chance that data about the strengths
and weaknesses of a product, service or program will not be interpreted fairly if the data are analyzed by
the people responsible for ensuring the product, service or program is a good one. These people will be
"policing" themselves. This caution is not to fault these people, but rather to recognize the strong biases
inherent in trying to objectively look at and publicly (at least within the organization) report about their
work. Therefore, if at all possible, have someone other than the those responsible for the product, service
or program to look at and determine research results.

Some Pitfalls to Avoid

Don't balk at research because it seems far too "scientific." It's not. Usually the first 20% of effort
will generate the first 80% of the plan, and this is far better than nothing.

There is no "perfect" research design. Don't worry about the research design being perfect. It's far
more important to do something than to wait until every last detail has been tested.

Work hard to include some interviews in your research methods. Questionnaires don't capture
"the story," and the story is usually the most powerful depiction of the benefits of your products, services,
programs, etc.

Don't interview just the successes. You'll learn a great deal by understanding its failures,
dropouts, etc.
 Research II / 3

Don't throw away research results once a report has been generated. Results don't take up much
room, and they can provide precious information later when trying to understand changes in the product,
service or program.

References:

http://www.crlsresearchguide.org/09_writing_state_of_purp.asp
https://explorable.com/research-population
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3117575/
https://managementhelp.org/businessresearch/analysis.htm

Communicate Research Findings Effectively

An oral presentation is more than just reading a paper or set of slides to an audience. How you
deliver your presentation is at least as important in effectively communicating your message as what you
say.

Preparing a Presentation

An effective presentation is more than just standing up and giving information. A presenter must
consider how best to communicate the information to the audience. Tips to create a presentation that is
both informative and interesting:

Organize your thoughts. Start with an outline and develop good transitions between sections.
Emphasize the real-world significance of your research.
Have a strong opening. Why should the audience listen to you? One good way to get their attention is to
start with a question, whether or not you expect an answer.
Define terms early. If you are using terms that may be new to the audience, introduce them early in your
presentation. Once an audience gets lost in unfamiliar terminology, it is extremely difficult to get them
back on track.
Finish with a bang. Find one or two sentences that sum up the importance of your research. How is the
world better off as a result of what you have done?
Time yourself. Do not Wait until the last minute to time your presentation. You want to know as soon as
possible if you are close to your time limit.
Create effective notes for yourself. Have notes that you can read. Do not write out your entire talk, use
an outline or other brief reminders of what you want to say. Make sure the text is large enough that you
can read it from a distance.
Practice, practice, practice. The more you practice your presentation, the more comfortable you will be
in front of an audience. Practice in front of a friend or two and ask for their feedback. Record yourself and
listen to it critically. Make it better and do it again.

Presenting Effectively

When you start your presentation, the audience will be interested in what you say. Use these tips
to help keep them interested throughout your presentation.

Be excited. You are talking about something exciting. If you remember to be excited, your audience will
feel it and automatically become more interested.
Speak with confidence. When you are speaking, you are the authority on your topic, but do not pretend
that you know everything. If you do not know the answer to a question, admit it. Consider deferring the
question to your mentor or offer to look into the matter further.
Make eye contact with the audience. Your purpose is to communicate with you audience, and people
listen more if they feel you are talking directly to them. As you speak, let your eyes settle on one person
for several seconds before moving on to somebody else. You do not have to make eye contact with
everybody, but make sure you connect with all areas of the audience equally.
 Research II / 4

Avoid reading from a screen. First, if you are reading from a screen, you are not making eye contact
with your audience. Second, if you put it on a slide, it is because you want them to read it, not you.
Blank the screen when a slide is unnecessary. A slide that is not related to what you are speaking about
can distract the audience.
Use a pointer only when necessary. If you are using a laser pointer, remember to keep it off unless you
need to highlight something on the screen.
Explain equations and graphs. When you display equations, explain them fully. Point out all constants
and dependent and independent variables. With graphs, tell how they support your point. Explain the x-
and y-axes and show how the graph progresses from left to right.
Pause. Pauses bring audible structure to your presentation. They emphasize important information,
make transitions obvious, and give the audience time to catch up between points and to read new slides.
Pauses always feel much longer to speakers than to listeners. Practice counting silently to three (slowly)
between points.
Avoid filler words. Um, like, you know, and many others. To an audience, these are indications that you
do not know what to say; you sound uncomfortable, so they start to feel uncomfortable as well. Speak
slowly enough that you can collect your thoughts before moving ahead. If you really do not know what
to say, pause silently until you do.
Relax. It is hard to relax when you are nervous, but your audience will be much more comfortable if you
are too.
Breathe. It is fine to be nervous. In fact, you should be – all good presenters are nervous every time they
are in front of an audience. The most effective way to keep your nerves in check – aside from a lot of
practice beforehand – is to remember to breathe deeply throughout your presentation.
Acknowledge the people who supported your research. Be sure to thank the people who made your
research possible, including your mentor, research team, collaborators, and other sources of funding and
support.
Sharing your work can help you expand your network of contacts who share your research
interests. For researchers, presenting can be an invaluable experience. We recommend discussing your
interest in sharing your research with your faculty advisor. They can help match your interests with the
appropriate venue.