A framework for the analysis of interview data from

multiple field research sites

Anne M. Lillis

Afzaal Ali

Business School, University of International

Business and Economics, Beijng, China.
 Anne Lillis is a Professor of Management Accounting
and Head Department of Accounting and Finance,
University of Melbourne, Melbourne, Victoria,
Australia.

BCom, University of Melbourne 1980

MCom, University of Melbourne 1990
PhD, University of Melbourne 1998

Anne is an Associate Editor of Journal of Management Accounting &

Research,
and a member of the editorial boards of Accounting, Organizations and
Society, Contemporary Accounting Research, Management Accounting
Research, Behavioral Research in Accounting, Accounting and
Finance, Abacus and the European Accounting Review.

 Anne researches and teaches in Management Accounting. Her research

interests focus on field studies of the design and behavioural influence of
performance management and control systems. Her teaching interests are in
Management Accounting, with a particular focus on capstone and case-based
teaching. 
 Summary of the article:
Quantitative and qualitative research are commonly considered to
differ fundamentally.

In this paper, the author basically tried to focus the issue with
qualitative data.

In general, papers reporting the results of research studies disclose

little detail regarding attributes of study design, analytical processes
and methods actually used by researchers.

The major focus of the paper is the application of a systematic

analytical protocol designed to encourage completeness and
impartiality in the collection and analysis of qualitative data.
Conti…

This paper describes in some depth the method choices and

analytical protocol used in a field study project.

To overcome this issue, the author follows a systematic analytical

protocol proposed by Miles and Huberman (1994) in their book
“Qualitative data analysis: An expanded sourcebook”.
Background of the problem:

Over the past 15 years, in the Accounting (Management Accounting)

literature researchers have been urged to shift their study of the role and
functioning of accounting to its natural contexts i.e. case/ field study
research.

While the call for fieldwork has been persistent, the penetration of this
research into top-ranking journals, particularly in the United States, has
been very limited.

Shields (1997) attributes the lack of publication of case/ field study

research to several factors, with lack of knowledge about how to do
good studies' leading the list.

Similarly Foster and Young (1997) argue that few studies meet the
criteria of high quality field research applied in other disciplines.
 Conti…
It may be that some of the problem lies with the way field study papers
are written.

Compared with surveys and laboratory studies, qualitative research, at

least in accounting, lacks an accepted language in which to
communicate study design parameters and the critical links between
design and credible research outcomes.

It seems that at least part of the difficulty in publishing field research

lies in convincing reviewers that the study is not only relevant and
interesting, but also trustworthy.
 Sections of the paper:
 Section-01: Already discussed

 Second-02: The description of the field study protocol

commences in this study with a discussion of the general research
question and implications for method and design choices.

 Section-03: and Section-04: Provide a brief overview of the study

design and method and an outline of the population and sample.

 Section-05: Outlines the general design characteristics of the

interview guide.
 Conti:

 Section-06: Details the analytical method applied to the qualitative

data.

 Section-07: Describes the way qualitative and quantitative data are

linked in this study to provide tests of convergent validity at several
levels.
Objectives of the paper:
This paper describes the application of qualitative study design
features that can enhance trustworthiness.

The paper examines the design of a systematic field study protocol.

Rather than an abstract discussion of method, the protocol is

described within the context of a particular study in order to link
explicitly the research question, research design and analytical
protocol.

Thus this paper focuses primarily on the design of an analytical

protocol that can support an audit trail from field data to the testing
or development of propositions.

An audit trail is a transparent description of the research steps taken from the start of
a research project to the development and reporting of findings.
 2. The motivation for field research:

 This paper describes part of a larger study of strategy and

performance measurement in manufacturing.

 While the larger study involved the collection and analysis of both
quantitative and qualitative data.

 This paper reports on the qualitative elements of the study.

 The study was motivated from contingency- type studies linking

strategy and management control system design and from the more
recent management accounting literature on strategic performance
measurement.
 Conti…
 Two issues are particularly relevant to the design of the qualitative
elements of the study:

 1st one:-
 The findings of contingency studies linking strategy with the use
and characteristics of management accounting controls have been
notoriously conflicting (Langfield-Smith, 1997).

 Past literature does not support predictions about the roles of any
specific performance measurement techniques in particular
strategic contexts.
 2nd one:-
 In the 1980s, the manufacturing side witnessed radical change in
context of competitiveness.

 In this situation, traditional performance measurement systems

were proved to be insufficient and to some extent become
irrelevant.

 As a response, some new solutions emerged in the form of the

Balanced Scorecard (Kaplan and Norton, 1992; 1996) and
Integrated Performance Measurement (Dixon et al., 1992).

 However, the emphasis shifted to promoting the development of

non-financial dimensions of performance such as quality and
customer responsiveness etc.
Financial Performance Measures:
Examples include:
Profitability
Revenue
Return on investment (ROI) 
Cash flow
Standard deviation or Variance
Coefficient of variation
Rate of return or return on investment
Working capital
Non- Financial Performance Measures:
Examples include:
Customer satisfaction
Employee satisfaction
Quality
Market share
Number of new products
Both of these issues suggest the need for an exploration of
competitive strategy
and its implications for accounting performance measurement
in practice.

Moreover, there is an apparent need to gain greater insight into

the empirical experience of variables
such as competitive strategy and the role of accounting
performance measures (Langfield-Smith, 1997).
 3. Overview of study design and method:

Data were collected using a semi-structured interview in combination with a

structured questionnaire administered to 50 managers in 36 manufacturing
firms in Victoria, Australia.

The use of a semi-structured interview schedule in combination with a

structured questionnaire aimed to derive the benefits of quantitative and
qualitative methods, and to apply appropriate methods to the questions of
interest.

Conventional statistical analyses were applied to data collected using the

structured questionnaire.

This paper focuses on the qualitative data collected during the semi-
structured interviews.
 Conti…
 The researcher used several approaches to limit biasness both during
the interview and in the analysis of transcripts.

 First, researcher designed an interview guide with the aim of ensuring

complete and consistent coverage in each interview session.

 Secondly, researcher in advanced developed a list of neutral questions

and queries in order to minimize interference.

 Fourth, a systematic auditable process developed by Miles and

Huberman (1994) was used.

 Finally, the use of structured questions provided quantitative response

from the respondents on established scales.
 4. The population:
 Time and cost considerations limited the sample of participant firms
to the population of manufacturers in the State of Victoria, Australia.

 The population was further restricted to relatively large

manufacturing firms with more than 200 employees.

 Researcher only consider firms that would have formal performance

measurement systems in place.

 A listing of 340 Victorian manufacturing firms with more than 200

employees was obtained in May 1994, from the Victorian State
Department of Business and Employment.
 The sample:
 Several features of the study design determined the sample size.

 The sample was required to be sufficiently large to be able to

observe patterns across firms.

 To conduct simple inferential statistical evaluation of results.

 However, the sample size was to be limited because the study

design required semi-structured interviews and a structured
questionnaire to be administered by a single researcher.
 Conti..
A sample size of 30-40 manufacturing firms was determined to be
appropriate in this context.

The firms were selected across industry categories with the aim of
obtaining the participation of firms of comparable size, and with a
variety of strategic orientations. .

In total, researcher contacted 55 profit centre managers. .

Out of 55, finally 36 managers were available to give feedback.

While 19 managers refused to participate in this study, and final
response rate was 65 percent.
Follow-up sites visits:

In order to confirm and find possible contradiction in the data

collected from profit centre managers, researcher selected 12
manufacturing managers from sample firms.

Researcher formally got permission from all of the profit centre

managers to pursue a follow-up interview with a manufacturing
manager.

The final sample consisted of 36 profit centre managers and 12

manufacturing managers.

The interviews were conducted between June 1994 and January

1995.
 5. Instrument design- the interview guide:

The study was conducted with a semi-structured interview and

structured questionnaire.

The interview guide used in this study is divided into four sections. For
example, the performance measures used, the constitution of cost
benchmarks and competitive priorities represent three separate
themes.

The interview protocol was developed from methods documented by

McCracken (1988) and Brenner, Brown and Canter (1985) based on
their qualitative research experiences.
 6. The qualitative analysis protocol:

In addition to the potential interviewer-induced bias in the

collection of qualitative data, the analysis of qualitative data is
subject potentially to significant bias as it relies on interpretations
and classifications imposed by the researcher.

Among the reasons why readers experience `lack of trust' in

qualitative studies lies in the absence of established techniques for
ensuring that data analysis is both complete and impartial.

On the other hand, the analysis of qualitative data is less structured.

Conti…

Readers are left questioning the extent to which propositions raised by

the researcher are supported by the data; whether potential alternative
propositions have been `overlooked‘; whether all cases were
systematically evaluated before propositions were raised; whether the
researcher saw only what they set out to see in the data; and whether
the process is reproducible.

The use of a systematic analytical protocol such as the Miles and

Huberman (1994) data display approach used here, enhances trust in
the results of qualitative analysis.
To analyzed and interpret qualitative data, researcher followed a
four steps qualitative analysis protocol.
The variable constitution of performance benchmarks was operationalized in the
qualitative data, and this variable was hypothesized to be related to strategy. The
structure of the matrix reflects this theoretical proposition.
Step 1: Transcripts were coded using the qualitative analysis package
QSR NUDIST. NUDIST was used in this study to code all of the raw,
unsummarised interview transcripts, by associating the sentences in the
transcript with one or more themes defined in a hierarchical coding
structure.
Each sentence was given a text-unit number in NUDIST. In this way, each
text unit is identified as relating to one or more pre- determined thematic
codes.
NUDIST is an acronym for Non-numerical Unstructured Data
Indexing, Searching and Theorizing. .

It is a qualitative analysis software package distributed by

Qualitative solutions and Research Pty Ltd, La Trobe
University, Bundoora, Victoria, Australia.
Step 2: A data sheet was prepared manually for each transcript. The text in
the transcripts was reconfigured and reported under conceptual headings
using NUDIST.
The reorganized transcripts were then summarized manually under similar
conceptual headings to those used in the coding scheme.
Step 3: The data sheets prepared for each transcript were then
combined in a multi-case matrix format.

For this study, these matrices and the analyses they supported were
focused on the variables to be intercorrelated.
Step 4: The content of the matrix was then built up case by case. Each
case was classified initially as `predominantly cost minimisation', `mixed'
or `predominantly differentiation' based on scores on the scaled strategy
variable.
 7. Linking qualitative and quantitative data:
 Thus, the findings were triangulated against the quantitative analysis
of participant-rated data on variables similar to those explored in the
qualitative data.
 For example, scaled responses were collected on strategy using
Porter‘s (1980) typology in a form similar to that used in survey-
based research (Govindarajan and Fisher, 1990).
 The scaled responses reflected an over- whelming importance of
differentiation strategies in the sample. This quantitative analysis was
compared with the elaborated responses on the firms' competitive
edge and strategic priorities.
 These quantitative and qualitative analyses `converged' to reflect a
similar strategic profile across the sample.

Data triangulation validates data and research by cross verifying the same
information
 Proposition:

The proposition emerged that firms experience greater difficulty

integrating performance criteria relating to customer responsiveness
with efficiency criteria than they did integrating quality and efficiency
criteria.
Retrospective evaluation of the method adopted:
 First, the study design achieved its purpose successfully.

 Beyond such re-examination of traditional contingency models,

the findings from this study raised new unhypothesised relations
that formed the foundation for theory building.

 The protocol used here establishes a disciplined approach to data

extraction and analysis that at least promotes completeness and
impartiality.

A retrospective evaluation may assess the degree to which goals were met.
Conti…

 She mentioned some researcher-related problems

specifically in qualitative study.

 There are practical matters such as scheduling too many

interviews too close together, not gathering sufficient
archival and other data while on site, getting too tired,
occasionally losing enthusiasm and concentration and thus
finding loose threads in interviews which should have been
followed up.
THANKS