Journal of Leisure Research

ISSN: 0022-2216 (Print) 2159-6417 (Online) Journal homepage: https://www.tandfonline.com/loi/ujlr20

Contextualizing reliability and validity in

qualitative research: toward more rigorous and
trustworthy qualitative social science in leisure
research

Jeff Rose & Corey W. Johnson

To cite this article: Jeff Rose & Corey W. Johnson (2020): Contextualizing reliability and validity
in qualitative research: toward more rigorous and trustworthy qualitative social science in leisure
research, Journal of Leisure Research, DOI: 10.1080/00222216.2020.1722042

To link to this article: https://doi.org/10.1080/00222216.2020.1722042

Published online: 10 Feb 2020.

Submit your article to this journal

Article views: 36

View related articles

View Crossmark data

Full Terms & Conditions of access and use can be found at

https://www.tandfonline.com/action/journalInformation?journalCode=ujlr20
JOURNAL OF LEISURE RESEARCH
https://doi.org/10.1080/00222216.2020.1722042

Contextualizing reliability and validity in qualitative

research: toward more rigorous and trustworthy qualitative
social science in leisure research
Jeff Rosea and Corey W. Johnsonb
a
University of Utah; bUniversity of Waterloo

ABSTRACT KEYWORDS
Issues of trustworthiness in qualitative leisure research, often demon- Epistemology; neoliberalism;
strated through particular techniques of reliability and/or validity, is ontology; paradigm; social
often either nonexistent, unsubstantial, or unexplained. Rather than science; trustworthiness
prescribing what reliability and/or validity should look like, research-
ers should attend to the overall trustworthiness of qualitative
research by more directly addressing issues associated with reliability
and/or validity, as aligned with larger issues of ontological, epistemo-
logical, and paradigmatic affiliation. In reviewing contemporary quali-
tative research methodologies, we present a variety of reliability and
validity techniques that might lead to increased trustworthiness in
analysis and representation of findings. Qualitative leisure scholars
are encouraged to align paradigmatic assumptions, theoretical orien-
tations, methodological practices, analytical techniques, and repre-
sentational practices to engage trustworthiness techniques that can
be assessed for quality and credibility. This conceptualization offers a
useful pedagogical model and supports common language of quali-
tative preferred practices, while providing space and openness for
growth, development, improvisation, and critique.

Bartender: What can I get y’all?

Editor: Gin, splash of tonic, lime.
Associate Editor (AE): Make it two. [Turning to Editor] So, that manuscript you sent me
this morning, what’s it all about?
Editor: Oh, you know, another “qualitative exploration” in a fairly interesting leisure
context, where they interviewed a bunch of folks and developed some themes.
AE: And?
Editor: And what?
AE: And … how good is it? Is it really ready for review? Some of the stuff I’ve been
getting lately doesn’t look as sophisticated as qualitative research should in our field in
2020. Did they engage deeply with the data? Is the connection to theory robust? Is it
persuasive? Did they position themselves in the context of the research? Do you get the
sense that the whole thing is trustworthy?
Editor: Hmm. Honestly, not usually, but seriously thanks for your service?

CONTACT Jeff Rose jeff.rose@utah.edu

ß 2020 National Recreation and Park Association
2 J. ROSE AND C. W. JOHNSON

In The Order of Things, Michel Foucault (1970) deploys his archeological method to
more fully unpack the ways various sciences, psychology and sociology specifically,
have materialized from historical periods with epistemological assumptions that help
determine contemporary social acceptability and legitimacy in the construction of
knowledge. His larger point is that we live in a sociopolitical time where the sup-
posed objectivity of scientific discourse is honored as more acceptable than other
interpretive ways of knowing (Parry & Johnson, 2007), or as recent posthumanists
have argued, even being (Kumm & Berbary, 2018). Consequently, this valuation of
scientism has substantial influence and effects on the ways in which social science
has positioned itself within an often unspoken hierarchy within institutions (univer-
sities, publishing outlets, etc.) and among academics. Most social scientists see them-
selves as being more empirical and relevant than the humanities, yet not as strong in
these regards as those in the natural sciences and/or mathematics (Flyvbjerg, 2001),
leading many social science researchers to use methodologies more epistemologically
aligned with the natural sciences to be perceived as more legitimate. But Foucault
imagines social science as owning its position within the knowledge production land-
scape more fully; rather than suffer from “physics envy” that “characterizes the par-
ade of behaviorism, cognitivism, structuralism, and neopositivism, [instead] he posits
a social science that takes values and power seriously” (Lather, 2006, p. 784).
Foucault (1970) positions the social sciences within a “cloudy distribution” (p. 347)
between the quantitatively calculable and the subjectivity-laden philosophical. These
liminal, interstitial spaces are where social science has the capacity to actualize and
contribute richly and more fully.
For social science to live up to its potential, though, it is incumbent upon both
researchers and consumers of research to demand high-quality products and processes,
in the epistemological orientation that has produced it (Morse, 2015). Throughout his-
tory, and even recently following the “qualitative turn” (e.g., Atkinson & Delamont,
2006, p. 170; Dixon-Woods et al., 2006, p. 39; Rennie, Watson, & Monteiro, 2002,
p. 188) in the social sciences, concerns about the quality or trustworthiness/legitimacy
of social science research borrowed heavily from the natural sciences and experimental
design research for direction. Borrowing from more positivist and quantitative
approaches, issues of objectivity, generalizability and, subsequently, reliability and valid-
ity had to be addressed for qualitative approaches to be deemed worthy within the
social sciences. To emphasize the point, generalizability, reliability, and validity were
positioned as “the holy trinity … , worshipped with respect by all true believers in sci-
ence” (Kvale, 1996, p. 229). Historically and perhaps even in many contemporary circles
of scholarship, qualitative research has felt the criticism from outsiders for its perceived
failure to “adhere to canons of reliability and validation” (LeCompte & Goetz, 1982,
p. 31). And while this devotion to reliability and validity remains paramount for many
qualitative social science discourses, there has also been a shift away from the absolut-
ism of canonical standards of reliability and validity associated with quantitative
research (Altheide & Johnson, 2013; Denzin, 2013; Johnson & Parry, 2015a; Schwandt,
1996; Seale, 1999a, 1999b). This shift aligns with epistemological transitions toward
interpretive, critical, and poststructural paradigms that both of us have experienced
across the course of our careers. Today, a broader concept of trustworthiness is often
 JOURNAL OF LEISURE RESEARCH 3

the goal for most qualitative inquiry (Johnson & Parry, 2015b), and we would continue
to advocate for that more strongly.
In this article, we provide a compilation and critique of various efforts to incorp-
orate reliability and validity into qualitative social science research, responding dir-
ectly to calls from qualitative methodologists (e.g., Morse, 2015; Roller & Lavrakas,
2015; Tracy, 2010). We begin by discussing various meanings of trustworthiness, reli-
ability, and validity. In this process, traditional recommendations for increasing reli-
ability and/or validity in qualitative social science research are presented to
demonstrate both successes and opportunities. Then, while not dismissing the
“politics of evidence” (Altheide, 2008, p. 137) that characterize many of these discus-
sions, we discuss a variety of techniques associated with demonstrating and poten-
tially increasing the trustworthiness of qualitative leisure research and scholarship.
These discussions of techniques are also influenced by engagements with critiques of
both reliability and validity, where paradigmatic alignment is an ultimate guiding
question for researchers. The purpose of our presentation of methodological impera-
tives to increase research trustworthiness (Morse, Barrett, Mayan, Olson, & Spiers,
2002) is to respond to the “tendencies and mandates to oversimplify research meth-
ods in qualitative studies” (McCoy, 2012, p. 762). More pointedly, qualitative
research in leisure sciences can and should provide stronger methodological and ana-
lytical rigor, as too often these issues are simply assumed in published qualitative
leisure scholarship or perhaps entirely overlooked in the research process. Qualitative
leisure research and scholarship has rarely explicitly addressed issues of trustworthi-
ness, reliability, and validity (see, for exception, Dupuis, 1999), and reliability and
validity in empirical qualitative studies are regularly addressed only in a passing
manner, if addressed at all. Such omissions may be due to decreased space in publi-
cations, failure to appreciate the conceptual and methodological importance, or not
actually engaging in these processes during qualitative research projects; regardless of
the reason(s), the omissions indicate a perceived lack of importance that should be
addressed. Here, we encourage leisure scholars to give more attention to the trust-
worthiness of conducting, analyzing, and reporting qualitative research, all of which
should be interconnected with issues of epistemology, ontology, paradigms, methods,
research questions, and so forth. While this discussion is taken up by many method-
ologists of qualitative research, it often seems lost in the training of more novice
qualitative leisure researchers. By incorporating various social science practices into
our collective qualitative research endeavors, we can strengthen the quality, import-
ance, and applicability of leisure-based qualitative social science research.

Understanding trustworthiness, reliability, and validity

Trustworthiness in qualitative research refers to the systematic rigor of the research
design, the credibility of the researcher, the believability of the findings, and applicabil-
ity of the research methods (Johnson & Parry, 2015a, Lincoln & Guba, 1985). It is the
overall impression of quality associated with a research endeavor. Harrison,
MacGibbon, and Morton (2001) suggested that trustworthiness is the key element to
maintaining the place of qualitative research in the academic world, and “there is a
4 J. ROSE AND C. W. JOHNSON

general consensus that qualitative inquirers need to demonstrate that their studies are
credible” (Creswell & Miller, 2000, p. 124). Without becoming bogged down by the
“criteriology” (Schwandt, 1996, p. 58; Sparkes & Smith, 2009, p. 491) that characterizes
swaths of positivist and postpositivist research paradigms (Parry, Johnson, & Stewart,
2013) and current neoliberal academic trends toward increased accountability (Rose &
Dustin, 2009), researchers in humanist1 qualitative inquiry should seek to more deeply
substantiate their analyses and claims. Studies that are more trustworthy are de facto
more likely to constitute research that makes a difference (Bochner, 2018; Roller &
Lavrakas, 2015) in leisure activities, perspectives, and/or phenomena, providing an
opportunity for leisure research to engage in the praxis of positive change (Rose,
Harmon, & Dunlap, 2018). We believe that increasing the trustworthiness of a qualita-
tive research study involves multiple aspects, including epistemological understandings,
the depth of literature reviewed and engaged, the appropriate theoretical positioning of
the argument, selection and deployment of the multiple and often conflicting data col-
lection/generation techniques and analytical procedures undertaken, the connection of
empirical material to larger theories and discourses, and the ways in which these aspects
of the research are interwoven with one another. An additional component of trust-
worthiness is addressing the reliability and validity of the research, through whatever
paradigm most aligns with a piece of research and scholarship. Echoing Gibbs (2007),
Creswell (2014) defines these concepts concisely: “Qualitative validity means that the
researcher checks for the accuracy of the findings by employing certain procedures,
while qualitative reliability indicates that the researcher’s approach is consistent across
different researchers and different projects” (p. 201). We briefly unpack these concepts
as elements of trustworthiness to provide more detail and further ground our recom-
mendations that humanist qualitative researchers strategically deploy them in the meth-
odological explanations of the products that result.
Reliability refers to the soundness of the research, particularly in relation to the
appropriate methods chosen, and the ways in which those methods were applied and
implemented in a qualitative research study. Reliability asks us to question the consist-
ency of the methodological process, hopefully remaining reasonably stable over time
and across researchers and/or methods engaged (Miles, Huberman, & Saldana, 2014).
Providing a justification of the methods used, as well as clarity in the analytical proce-
dures, increases a sense of reliability of a study. Reliability also addresses the consistency
and clarity associated with the actual conduct of the research, thereby increasing the
likelihood that other researchers could not only discern but also undertake many of the
research methods described (Creswell, 2013). Questions we might ask to address issues
of reliability might include the following: Could a reasonable researcher conduct a simi-
lar research project based upon the description provided? To what degree is this
research project replicable? If the research project were conducted again, would similar
results and analyses occur?
Validity refers to the process of determining the fidelity (sometimes understood as
accuracy) of the findings from the standpoint of the researcher, the participants, and/or

1
We distinguish humanist social science in this article as those (often unacknowledged) ontological positions that
privilege hierarchies associated with human dominance, supremacy, and agency as sites and sources for knowing, as
opposed to posthumanism, actor-networks, and new materialist ontologies (see Berbary, 2015).
 JOURNAL OF LEISURE RESEARCH 5

the consumers of the research (Creswell & Miller, 2000; Lincoln, Lynham, & Guba,
2013). Like reliability, validity is a contested term and is derived from a traditional
quantitative construct, placing it in a tenuous position for qualitative researchers,
regardless of their ontological or paradigmatic affiliations. Many qualitative researchers,
like Corey eschew the notion of validity altogether, and prefer verisimilitude (the degree
to which an analysis aligns with reality) or a persuasively written account (Miles et al.,
2014). Cho and Trent (2006) place validity into two basic types: transactional, or the
validation of themes and interpretations; and transformational, the ability of research to
make change in people’s lives. As demonstrated in the validity techniques below, para-
digmatic assumptions inform whether transactional or transformational validity
should be sought. Some qualitative researchers, also questioning or outright rejecting
the quantitative underpinnings of validity (cf., Lincoln et al., 2013), advocate for deep
understanding, whether that be descriptive, interpretive, theoretical, or evaluative
(Wolcott, 1990).
In sum, trustworthiness, which includes issues commonly though not exclusively
associated with traditional markers of reliability and validity, is a vital component of
research across the natural sciences, social sciences, and humanities. Within these dis-
ciplinary identifications, the overall quality of a research endeavor should be maintained
and articulated across a variety of ontological, epistemological, and/or paradigmatic
affiliations (Jackson & Mazzei, 2012); leisure researchers doing humanist qualitative
social science should be just as concerned with trustworthiness as are chemists under-
taking an experimental design. And while concern is likely shared across disciplinary
and epistemological variations, the techniques for supporting trustworthiness
vary widely.

Techniques for trustworthiness

Depending on the particular approach one takes, qualitative researchers have previ-
ously sought to increase trustworthiness through validity and/or reliability in a num-
ber of ways. Some have used techniques that quite closely parallel positivist
approaches and/or the techniques of quantitative research, addressing issues like
internal validity (whether the analyses represent reality), external validity (whether
the analyses compare to other populations or settings), reliability (whether the study
is replicable), and objectivity (whether the analyses depend upon the researcher;
LeCompte & Goetz, 1982). Other qualitative researchers have focused on issues like
credibility (analyses are believable), transferability (analyses can be transferred to
other contexts), dependability (analyses are consistent and could be repeated), and
confirmability (analyses are supported by the data; Lincoln & Guba, 1985; Patton,
2014), demonstrating the nuances that come with differentially framing similar con-
cepts. More recent manifestations of reliability and validity have encouraged the use
of crystallization (Richardson & St. Pierre, 2005) and authenticity and integrity of
the research (Whittemore, Chase, & Mandle, 2001). Perhaps most pointedly, Miles
et al. (2014) ultimately want to address the “truth value” (p. 312) of a qualitative
study’s findings and analyses. Further, researchers, like us, who operate from critical,
poststructural, and similar epistemologies are often skeptical of reliability and
6 J. ROSE AND C. W. JOHNSON

validity, if these terms even apply at all.2 While discursive framings continue to
expand with ongoing debates about qualitative research trustworthiness (e.g., Lincoln
et al., 2013; Marshall & Rossman, 2016), conceptually these various efforts are all
concerned with increasing (and demonstrating) the quality of the research.
Specific methodological techniques necessary for supporting trustworthiness of quali-
tative research through reliability and/or validity are dependent upon a variety of con-
textual factors (Morse et al., 2002). Based on the work of Ellingson (2009), Marshall
and Rossman (2016) present qualitative social science research as continuum ranging
from “art/impressionist” to “science/realist” (pp. 45–46), and depending on where one’s
research project is positioned across this spectrum will heavily influence what consti-
tutes “quality” in research and the criteria it should be judged by. As qualitative
research is often concerned with “methods and paradigms that reflect a high interest in
ontologies and epistemologies” (Lincoln et al., 2013, p. 199), these orientations can help
direct researchers toward understandings of trustworthiness. Depending upon a
researcher’s paradigmatic position on humanist qualitative inquiry, ranging across a
spectrum of positivist, postpositivist, critical, constructivist, and participatory (Table 1),
researchers may engage a variety of techniques that approach what efforts toward reli-
ability and validity seek: “goodness” or criteria of quality (Lincoln et al., 2013, p. 205).
Without delving into an extensive discussion of ontologies, epistemologies, and resulting
methodological choices, the point is to emphasize that subtle differences in these com-
plex aspects of a study should inform notions of trustworthiness. So, while postpositivist
qualitative researchers might be highly interested in demonstrating a study’s internal
validity using multiple techniques (i.e., large sample, use of an interview guide, multiple
researchers), a more critical orientation might determine the quality of the research by
its ability to act as a catalyst for social or political change (i.e., increases access to
resources by marginalized groups). Across these orientations, judgments of the
research’s “goodness” are attempts to better understand the systemic rigor, quality, and
ultimately the usefulness of the research, even if such criteria are increasingly fluid and
emergent (Lincoln, 1995).
As researchers, we can incorporate numerous reliability and validity techniques
throughout the research process. Although often considered as single-event occurrences,
reliability and validity processes should occur throughout the qualitative research pro-
cess, as should concerns and expositions about researcher subjectivity and reflexivity
(Johnson & Parry, 2015b). Qualitative research often unfolds in an iterative manner,
with less distinction between processes of data collection and data analysis (Miles et al.,
2014), and researchers should consider the iterative, ongoing, in-process incorporation
of the reliability and validity techniques outlined below, depending upon paradigmatic
alignment. The following techniques for addressing reliability and validity, unless other-
wise noted, have been compiled and adapted from Lincoln and Guba (1985, pp.
301–320), Creswell (2013, pp. 250–253, 2014, pp. 201–203), Marshall and Rossman
(2016, pp. 46–48), Patton (2014, pp. 670–682), and/or Miles et al. (2014, pp. 312–313).
We conclude each section with a critique that should be noted for qualitative
researchers.

2
For a detailed history of the development of qualitative engagement with techniques of validity, see Creswell (2013,
pp. 244–245).
Table 1. Paradigmatic alignment with humanist qualitative trustworthiness approaches (adapted from Lincoln, Lynham, & Guba, 2013, pp. 204–205)6.
Positivism Postpositivism Critical Theories Constructivism Participatory
Ontology Naïve realism: “real” objective Critical realism: “real,” Historical realism: realities Relativism: local and specific Participative reality:
reality that is conditional reality but only shaped by social, political, co-constructed realities subjective-objective reality,
apprehendable imperfectly and cultural economic, ethnic, co-created by mind and
probabilistically and gender values; given cosmos
apprehendable crystallized over time

Goodness or quality criteria
Note. For a discussion on paradigmatic approaches specific to leisure studies, see Parry et al. (2013).
Conventional benchmarks of “rigor”: internal and external Historical situatedness;
validity, reliability, and particular types of objectivity erosion of ignorance and
misapprehensions;
action stimulus
Trustworthiness and
authenticity including
catalyst for action
Congruence of experiential,
presentational,
propositional, and practical
knowing; leads to action
to transform the world in
the service of human
flourishing
JOURNAL OF LEISURE RESEARCH
7
8 J. ROSE AND C. W. JOHNSON

Reliability techniques and critiques

One of the traditional questions considering trustworthiness of qualitative research asks
us whether the findings would be similar if the same questions, with the same people,
and with the same basic situational context were engaged in another study. Increasing
reliability can be achieved through a variety of consistencies that demonstrate a study’s
rigorous and systematized nature. Recordings and/or transcripts can be inspected to
account for possible mistakes (Gibbs, 2007). There should be consistent documentation
of methodological procedures, aided by the development of detailed study protocols and
a database, so that others can follow similar procedures (Yin, 2012). Themes or codes
that are developed during analysis should be clearly defined so that meanings of the
codes are consistent, regardless of the number of analysts involved in the project (Miles
et al., 2014; Saldana, 2013). Demonstrating these codes during publication and/or pres-
entation increases the reliability of the deductive analysis in the research project.
For qualitative research projects involving multiple researchers and/or analysts, there
is an increased importance on effective communication between team members
(Creswell, 2014). Regular, documented team meetings create settings where shared ana-
lysis can occur and where questions can be clarified (Saldana, 2013). Codes that are
developed by separate researchers should be cross-checked by comparing results that
are independently derived (Gibbs, 2007). There are also times where a single know-
ledgeable coder analyzes all transcripts or data once the coding scheme has been estab-
lished and is then supported (or not) by secondary coders who can increase reliability
(Campbell, Quincy, Osserman, & Pedersen, 2013). Intercoder agreement, interrater reli-
ability, or cross-checking in qualitative research requires that data analysts agree that
particular passages in a text are aligned with particular codes (Armstrong, Gosling,
Weinman, & Marteau, 1997). Reliability, in this instance, is the process where a separate
coder would likely code the text with the same or similar codes.3 It may be helpful to
standardize the units of text to be coded and then reduce coding errors among multiple
coders by measures of intercoder reliability (Armstrong et al., 1997; Campbell et al.,
2013). Miles et al. (2014) suggest that, during the analysis process, “intercoder agree-
ment should be within the 85 to 90% range, depending on the size and range of the
coding scheme” (p. 85). Engaging and reporting these processes support the reliability
of the results, as well as potentially the stability of responses across multiple coders.
We feel strongly that subjectivity and reflexivity are paramount in qualitative research
processes (Johnson & Parry, 2015b). While positivists and even postpositivists might use
the term “bias,” subjectivity is the views, experiences, and positions we bring with us
into our research endeavors. Slightly different from subjectivity, reflexivity is the
ongoing process of incorporating personal reflections concerning our subjectivities
within the context of theoretical and paradigmatic considerations across the research
project, from design to dissemination. While all qualitative researchers should incorpor-
ate reflexivity into the design, implementation, analyses, and representation of the
research, highlighting these issues is increasingly imperative for researchers operating
under interpretivism or other critical perspectives; in these paradigms, subjectivity and

3
A variety of computer-assisted qualitative data analysis software (CAQDAS) can help determine levels of interrater
reliability, or the consistency of the coding process.
 JOURNAL OF LEISURE RESEARCH 9

reflexivity are vital when considering issues of reliability, which questions the replicabil-
ity of a research project. Incorporating subjectivity and reflexivity means that it is very
unlikely that different researchers might arrive at similar results, and this discrepancy is
in fact a strength of qualitative research. Different researchers providing differing analy-
ses may provide the insight, engagement, and nuance that is necessary to better under-
stand a particular social science phenomenon. For these reasons, many qualitative
researchers are completely eschewing reliability as either a feasible or desirable feature
of their inquiry; one’s paradigmatic affiliations should guide such decisions.

Validity techniques and critiques

We have noted that validity, or trying to understand the “accuracy” of the research
findings, has gained increased attention in humanist qualitative scholarship (Cho &
Trent, 2006; Davis & Lachlan, 2017; Lincoln et al., 2013). In qualitative research, there
are multiple, often highly specified types of validity that align with other aspects of the
research project.4 Considering the diversity of these options, there is likely a form or
forms of validity that work for a particular qualitative research project, regardless of the
project’s paradigmatic affiliation(s) (Altheide & Johnson, 2013). Listed below are a var-
iety of trustworthiness techniques we identified for qualitative research that are com-
monly associated with issues of validity. While a checklist of validity tests may be
helpful in a qualitative study (Maxwell, 2013), it seems prudent to echo recommenda-
tions that some of the following techniques be engaged and reported during a qualita-
tive study,5 although these recommendations should account for various paradigmatic
assumptions and disciplinary conventions. Further, each of the following techniques
may be more applicable, warranted, expected, and necessary in some paradigmatic
approaches than others. For example, if studying men’s rights groups from a feminist
perspective, you would not necessarily seek confirmatory legitimation from your partici-
pants on the critique of their discourses and behaviors. So, it is important that your
strategies align with your overall purpose and research questions. Despite this example,
the most popular form of building trustworthiness is member checking.

Member checking
Member checking involves sharing (anonymous) collected data—often in the form of a
draft report—with research participants and receiving their feedback about that process.
These include “opportunities for the researcher to test her findings, interpretations, and
explanations within the culture she is studying” (Davis & Lachlan, 2017, p. 179).
Through personal interviews, focus groups, emails, or otherwise, participants decide to
what degree preliminary codes, themes, texts, participant quotes, cultural description, or
a grounded theory represent their own sense of the question or phenomenon. We
would like to note that this process is distinct from transcript verification, which is

4
Types of validity include successor validity, catalytic validity, interrogated validity, transgressive validity, imperial
validity, simulacra/ironic validity, situated validity, voluptuous validity, and hyphenated validity. While it is beyond the
scope of this manuscript to describe each of these techniques, more detail can be found in Altheide and
Johnson (2013).
5
Creswell (2013) recommends at least two techniques be conducted.
10 J. ROSE AND C. W. JOHNSON

more technical and objective in participants’ agreement that a textual representation of

an interview is, in fact, what was communicated. Member checks are usually conducted
with some study participants but may also include others within a culture who were not
involved in the original data collection process, further increasing the credibility of the
research (Davis & Lachlan, 2017). Member checking is based upon traditionally con-
structivist views of validity (Seale, 1999a); in this sense, “data must be continually
proven” (Caretta, 2016, p. 312), although the presentation of discrepant cases (see dis-
confirmation below) also opens the data to multiple interpretations. To highlight the
active roles of participants in contributing to the research, member checking can be
engaged as a process of “catalytic validity” (see below) where researchers involve partici-
pants’ processes of correcting and reorienting researchers and the developed qualitative
data to produce more representative analyses of social phenomena (e.g., Caretta, 2016).
Member checking has limitations, including that research participants may not necessar-
ily recognize their own perspectives reflected in themes analyzed by the researcher(s).
As we mentioned earlier, a research project’s fidelity to participants’ perspectives should
be aligned with the paradigmatic approach of the research, where some paradigms are
more interested in direct accounts of participants, and others are more interested in
how these accounts are leveraged. In most cases, an honest reporting of the participants’
responses is more important than agreement or disagreement with the findings. Indeed,
the latter often provides room for additional analysis and interpretation on the part of
the researcher and can frequently enrich and complexify the findings and discussion.

Triangulation
Triangulation involves recognizing multiplicity and simultaneity of cultural frames of
reference (Atkinson & Delamont, 2008), providing a plurality of techniques to best
ensure accurate description and presentation of a given situation. The idea is that by
addressing a phenomenon from multiple directions, researchers can more clearly and
accurately “locate” that phenomenon. Decades ago, Denzin (1978) identified how at
least four different types of triangulation might contribute to increasing validity:
Different data collection methods can be compared for consistency; data sources within
the same method can be assessed for consistency; multiple analysts can review findings;
and multiple paradigms or theories can be used to interpret the data. Further, multiple
types of analyses can be employed to promote rigor of the study (Leech &
Onwuegbuzie, 2007). The type of triangulation necessary is dependent upon a variety of
ontological, epistemological, theoretical, and methodological considerations. While lit-
erature expresses the need for triangulation (i.e., Farmer, Robinson, Elliott, & Eyles,
2006), there is little leisure scholarship that operationalizes the actual process (for excep-
tion, see Decrop, 1999). Qualitative codes, themes, and overall analyses can be more
coherently justified based on a converging coherence from multiple sources, increasing
the validity of the study.

Critical reflexivity and subjective positionality

Being critically reflexive enables researchers to better interrogate assumptions that are
associated with our own subjectivities, particularly those that play out in the research
 JOURNAL OF LEISURE RESEARCH 11

process (Miller, Grimwood, & Arai, 2015; Stuart, 2017). Further, many of these subject-
ive positions require interrogation prior to the research formulation or implementation
(Johnson & Parry, 2015b). Questions of self, politics, intent, motivation, and others are
paramount in this process. While reflexivity has long been established as a key compo-
nent in qualitative research (since the researcher is the research instrument), validity of
the overall study can be strengthened with thoughtful, insightful articulation of the ways
in which researchers’ subjective positionalities influence all aspects of the research pro-
cess, from subject matter to methods to analysis to representation of the findings. Our
social markers, including race, class, gender, culture, history, and socioeconomic origin
color our studies and should be identified and acknowledged as appropriate. In more
positivist terms, this process might look like identifying the biases that researchers bring
to the matter, even as such incoming positionalities are celebrated and unpacked in
both interpretivist and critical paradigms.

Rich, thick description

Although regularly employed by ethnographers like us, constructing narratives and anal-
yses that provide nuanced, contextual detail of social, political, or economic phenomena
increases the sense of a qualitative researcher’s overall embeddedness and awareness of
the “field” in which they are studying. Increased attention to the richness and detail of
the data provides a sense of realism for readers, increasing the sympathy for participant
perspectives or the more contextualized understanding of a phenomenon. These
descriptions may come from the researcher/author, but they may also be extracted from
participant quotes, narratives, or perspectives (e.g., Furman, Lietz, & Langer, 2006). The
type and degree of description likely is dependent upon paradigmatic and methodo-
logical considerations and traditions. For example, an interpretivist study will likely
desire more description than a postpositivist one, and an ethnography will demand
much more thick and rich description, than say, a grounded theory study or a
phenomenology.

Catalytic validity
Researchers grounded in more critical paradigmatic approaches might judge the validity
of their work more on its ability to transform sociopolitical situations of concern, often
focusing on an explicit agenda of social justice. Paradigmatically, researchers see direct
interplays among education, research, and advocacy, or research as praxis (Lather,
1986a, 1986b). In this way, social change toward more humane, tolerant, progressive,
just politics is an explicit outcome of the research, and the degree to which that change
occurs is a reflection of the research project’s catalytic validity. What the study does,
what it accomplishes, for participants, researchers, and consumers, is of vital importance
when considering catalytic validity. We would argue that researchers should interrogate
the issues of working for communities, and working with communities, and subsequently
unpacking the implicit power dynamics associated with these positions.
12 J. ROSE AND C. W. JOHNSON

Crystallization
In 2007, Laurel Richardson introduced us to the metaphor of a crystal as useful for
thinking about the ways in which hidden assumptions within a research project are
reflected and refracted in the process of the research (Ellingson, 2009; Richardson,
1997; Richardson & St. Pierre, 2005). Critiquing the fixed, rigid structure of the tri-
angle (in triangulation), Richardson (1997) argues that crystals are “prisms that
reflect externalities and refract within themselves” (p. 92). The researcher’s self-cri-
tique is mapped onto the external findings and analyses, providing multiple perspec-
tives, colors, and refractions, thereby increasing validity. We feel this is an extremely
useful metaphor, especially for those doing qualitative research beyond postpositi-
vist paradigms.

Searching for disconfirmation (negative case analysis)

Do inconsistencies in research mean it is wrong? No. Researchers should search for the-
ories, data, or discrepant information that run counter to themes or analyses developed
in the research. Further, they should seek alternative explanations for the ways in which
their research frames and understands the phenomenon (Maxwell, 2013). By presenting
data and evidence (i.e., negative cases) that both support and contradict general per-
spectives of a theme, researchers increase the validity of the claims they make through
their research. The social and political world is a complex landscape, and acknowledging
these variations likely presents a more realistic (and believable) account of the phenom-
enon of interest.

Peer debriefing
It is often helpful in qualitative research to engage with a person who is methodologic-
ally and analytically adept, but not embedded in the research topic or process as much
as the primary researcher. This person (or persons) assists by questioning methodo-
logical practices, analytical techniques or frames, and overall clarity of the research
endeavor, contributing to the resonance of the research. Graduate students, in particu-
lar, may find this process by engagement with research supervisors and committee
members. However, everyone could benefit through the use of writing groups, reading
groups, and collaboration. We have found that peer debriefing may expand the develop-
ment of various analytical schemes (including coding), increase methodological compe-
tencies, and support trusted relationships already existing between the researcher and
the peer (Spall, 1998). In fact, Maxwell (2013) suggests that one validity check might
include soliciting feedback from those familiar with the setting and from strangers, a
point that is similar to the techniques and rationales for peer debriefing. Like member
checking and other techniques, the usefulness of peer debriefing likely depends upon
the paradigmatic approach, as some paradigms place more value than others do in
external input into analyses.
 JOURNAL OF LEISURE RESEARCH 13

Prolonged engagement
While the amount of time spent “in the field” has long been a marker of ethnographic
rigor (Borneman & Hammoudi, 2009; Low, 2014), we think nearly all qualitative
research can increase its validity with increased engagement (Lather, 1986b). Like the
thick, rich description, prolonged engagement provides support for the researcher’s
(implicit or explicit) claims to a greater depth of understanding of the phenomenon in
question. Extended time obviously increases the likelihood of gathering more data,
increasing the validity of the findings, although researchers increasingly acknowledge
that intensity of engagement may be a better marker than strictly chronological engage-
ment (Pink & Morgan, 2013). Prolonged engagement might be more useful for particu-
lar paradigms (critical, participatory) and methodologies than others. For example,
ethnography demands it, whereas a case study does not.

External auditor and audit trail

An external auditor can review the entire research project, including not only the find-
ings and analyses but also the process through which the findings were developed. We
have served in these capacities on many occasions. Unlike a peer debriefing partner, an
external auditor is likely unfamiliar with the researcher and possibly unfamiliar with the
research overall project. The auditor might read final reports or might examine specific
aspects of the project: field notes, interview transcripts, videos of focus groups, etc. This
person might also look at larger aspects, such as the alignment between the research
questions and the data that were produced. Perhaps presumed by the presence of an
external auditor is the existence of an audit trail. An audit trail is documentation that
provides support for the techniques undertaken throughout the research process,
increasing the perception that the data gained from these techniques are more represen-
tative of the phenomenon under consideration. This final aspect highlights the intercon-
nectedness of reliability and validity in traditional approaches to qualitative research.
The listed techniques for reliability and validity are some of the many options that
qualitative researchers have for increasing the trustworthiness of our research. Table 2
(below) provides a continuation of Table 1, where we describe how trustworthiness
techniques range across positivism, postpositivism, critical theories, constructivism, and
participatory approaches. Some of the reliability and validity techniques align firmly
with particular approaches, while some paradigms might eschew the discursive framing
of reliability and validity as too historically and contemporarily rooted in positivist and
postpositivist thought and practice. Regardless of how these more critical approaches
might term these techniques, our contention is that there remains a need to strive for
increased trustworthiness in qualitative leisure research.
Although Table 2 suggests some alignments between paradigms and trustworthiness
techniques, it does not advocate that any particular technique or set of techniques are
preferred. Rather, in Table 2 we point to the necessity to carefully consider how particu-
lar approaches are more or less aligned with particular trustworthiness techniques that
were briefly described previously. While some of these techniques are more often associ-
ated with a particular approach, there is much fluidity and hybridization that can be
incorporated into these alignments. It is unlikely that any one of these techniques
14 J. ROSE AND C. W. JOHNSON

Table 2. Paradigmatic alignment of trustworthiness techniques.

simply “works” with all paradigmatic approaches, or even with a particular paradigmatic
approach; rather, researchers may have to bring in multiple techniques, incorporate
existing techniques partially, and/or create new, potentially hybridized techniques to
support increased trustworthiness in findings and analyses. Considerations of ontology,
epistemology, paradigms, and methodological and disciplinary traditions should remain
paramount when considering trustworthiness.
Other factors contribute substantially, including the strength of relationships with the
participants and the “field” of research. As Dreher (1994) explains:
[The] quality, validity, and reliability of the data are grounded in the skill of the
investigator to establish relationships with informants. … They are achieved through an
extended, trusting, and confidential relationship between investigator and informants rather
than through the establishment of the psychometric properties of research instruments
(p. 286).
Beyond this always contested researcher–researched relationship, the ways in which
qualitative researchers establish reliability and validity are not only dependent upon
paradigmatic, methodological, and localized contextual factors of the study but also
upon themselves, where a high degree of reflexivity is required to consider the ways in
which we account for ourselves in the research process (Altheide & Johnson, 2013;
Miller et al., 2015; Stuart, 2017). The choice of the topic and the interest in the topic,
combined with methodological approaches and paradigmatic affiliation, should influence
the ways in which researchers engage with issues of trustworthiness.
We have witnessed the history of the development of trustworthiness in qualitative
research and how it parallels the development of the use and acceptance of qualitative
methods as worthy endeavors in the social sciences, as well as those in leisure studies.
 JOURNAL OF LEISURE RESEARCH 15

Reliability and validity were taken and amended by more positivist social scientists
(using quantitative methods) from the natural sciences, and then subsequently taken
and amended by postpositivists (both quantitative and qualitative researchers) from
positivist social scientists. We understand that reliability and validity were vital bridges
built to demonstrate the rigor and depth of qualitative research. However, as social sci-
ences have expanded from strictly positivists to now include postpositivists, interpreti-
vists, critical theorists, and a plethora of other paradigms, many of the traditions of
reliability and validity among qualitative researchers can now transition to practices of a
broader notion of trustworthiness. Contemporary qualitative researchers should seek to
increase the overall trustworthiness of their research and therefore may choose to
engage with some of the techniques of reliability and validity discussed here.
Debates will likely continue within and beyond qualitative research about what
aspects should constitute the criteria for assessing the trustworthiness of a study, who
should make these determinations, and the attendant discussion of what constitutes
“evidence” (Cho & Trent, 2006; Marshall & Rossman, 2016). In fact, many post-qualita-
tive researchers have begun to question the value of what we argue for here (cf., Kumm
& Berbary, 2018). All of these dynamic considerations constitute worthy conversations,
ones in which leisure researchers should more actively engage. As there are increasing
calls for evidence-based practice, outcome measures, program assessments, critical social
change, and other paradigm-dependent accountability schema, we argue that the diverse
manners in which researchers choose to justify or legitimize their work become increas-
ingly imperative (Altheide & Johnson, 2013).

Toward increased trustworthiness of qualitative leisure research

Humanist qualitative researchers tend to ask questions revolving around issues of mean-
ing, understanding, experiences, and perspectives, questions that are paramount across
leisure research methodologies, traditions, and topics. These issues, when addressed
through qualitative research methodologies, are nearly always highly subjective, requir-
ing human interpretation and reasoning to help make sense of the data. Therefore, there
exists a seemingly a priori tension between the subjective nature of qualitative research
and the perhaps perceived need to justify one’s analyses through a variety of external,
often more positivist paradigm of accountability (Cho & Trent, 2006; Creswell & Miller,
2000; Jackson & Mazzei, 2012; LeCompte & Goetz, 1982; Rossman, Rallis, & Kuntz,
2010; Schwandt, 1996; Wolcott, 1990). However, the demands of seeking trustworthiness
in qualitative research are not entirely external; many qualitative researchers, internally
and self-reflexively, want to ask of themselves, “Did we get it right?” (Stake, 1995, p.
107) or “Did we publish a ‘wrong’ or inaccurate account?” (Thomas, 1993, p. 39).
While such questions are unlikely to produce definitive answers, they are effective in
encouraging a careful consideration of the ways in which qualitative researchers can use
reliability and validity to increase the overall trustworthiness of their research. The
point, then, is that it is incumbent upon qualitative researchers in leisure studies and
beyond to acknowledge both that this tension exists, on one hand, and on the other
hand, the ways in which the researchers chose to engage with and address this tension
16 J. ROSE AND C. W. JOHNSON

between subjective experiences and objective criteria of assessment, if such is

even possible.
As qualitative leisure researchers, we should more consciously and assertively conduct
trustworthy research and report the nature, results, and textures of these techniques in
their scholarly products. Practicalities of this process require attention, though, as edi-
tors and publishers regularly pressure manuscripts for page limits and word counts, and
the trustworthiness processes described here require clarity and explanation. We
encourage researchers to present these processes clearly but concisely and to treat trust-
worthiness as an integral component of any research endeavor. Using conventional
techniques and appropriate references, this process might result in only a few additional
sentences; inventive and hybridized trustworthiness efforts might require more explan-
ation, but the addition of this space is worth the increased rigor and quality. As
researchers, reviewers, advisors, and other consumers, we should ask ourselves a series
of critical questions throughout the duration of qualitative research processes: How
does this research align throughout, from idea initiation to methods conducted to analy-
ses to study closeout to representation of the findings? In what ways does this research
correspond to reality? In what ways does this research support a particular agenda?
How does this research ensure its “goodness”? How does this research make sure that
quality qualitative work is being conducted, and how are these processes represented in
subsequent scholarship?
In our experience, across a variety of ontological and epistemological positionings
(Jackson & Mazzei, 2012), there remains a problematic lack of acknowledgement of the
dominant role that scientific discourse plays in our knowledge production systems. A
number of scholars from a variety of disciplines have pushed back against this tide.
Interpretive researchers from across “the empowerment discourses: critical construction-
ists, feminists, critical pedagogy and performance studies, oral historians, critical race
theory, interpretive interactionists, etc. … seldom use terms like ‘validity’ or
‘reliability’” (Denzin & Giardina, 2009, p. 25). However, despite the rise of a number of
critical and poststructural ways of knowing, it remains that most epistemological para-
digms that engage qualitative research acknowledge “that while knowledge is not object-
ive, scientific validity is an ideal to strive toward” (Caretta & Riano, 2016, p. 259,
emphasis in original). Overall trustworthiness remains a stringent ultimate that most us
as scholars seek, regardless of the problematic assumptions that may be embedded
within. Scholars from leisure sciences should also hold ourselves, our colleagues, and
our students to similarly rigorous accepted practices, while also inventing, innovating,
and hybridizing techniques and practices that further support the trustworthiness of
our research.
Foucault’s (1970) argument in The Order of Things ran contrary to this scientism,
emphasizing that the social sciences’ strengths lay in their abilities to move beyond the
depoliticized strictures that accompanied Enlightenment thinking. There are multiple,
and often contradictory positionings of the social sciences within the academy.
Returning to the notion that those of us in the social sciences, and specifically those of
us conducting qualitative inquiry within the social sciences, suffer from “physics envy,”
we would do well to remember that our colleagues in physics departments across cam-
pus rarely operate “by imagining the universe as a closed and determined system,
 JOURNAL OF LEISURE RESEARCH 17

let alone the fantasy that human beings can intervene to make it operate efficiently uti-
lizing cost/benefit analyses according to free-market logics” (McCoy, 2012, p. 762).
However, this scientism-based envy is a contemporary part of academic culture, whether
supported by intellectualism, cultural hegemonies, or the logics and imperatives of neo-
liberalism, both broadly across our political economies (Harvey, 2005) and within aca-
demia at large (Dowling, 2008), and leisure research, more specifically (Rose & Dustin,
2009). Neoliberalism, in this way “provide[s] regulatory regimes, conceptual and mater-
ial pathways to channel and limit certain ways of working with complexity in qualitative
research” (McCoy, 2012, p. 762). But it is the necessary complexity of qualitative
research, like the political and politicized natures that Foucault imagined in the social
sciences, that serves as the intellectual and political strengths of these endeavors.

ORCID
Jeff Rose http://orcid.org/0000-0003-3171-7242

References
Altheide, D. (2008). The evidentiary narrative: Notes toward a symbolic interactionist perspective
about evidence. In N. Denzin & M. Giardina (Eds.), Qualitative inquiry and the politics of evi-
dence (pp. 137–162). Walnut Creek, CA: Left Coast Press.
Altheide, D., & Johnson, J. (2013). Reflections on interpretive adequacy in qualitative research. In
N. Denzin & Y. Lincoln (Eds.), Collecting and interpreting qualitative materials (pp. 381–412).
Los Angeles, CA: SAGE.
Armstrong, D., Gosling, A., Weinman, J., & Marteau, T. (1997). The place of inter-rater reliability
in qualitative research: An empirical study. Sociology, 31(3), 597–606. doi:10.1177/
0038038597031003015
Atkinson, P., & Delamont, S. (2006). Rescuing narrative from qualitative research. Narrative
Inquiry, 16(1), 164–172.
Atkinson, P., & Delamont, S. (2008). Analytic perspectives. In N. Denzin & Y. Lincoln (Eds.),
Collecting and interpreting qualitative materials (pp. 285–311). Los Angeles, CA: SAGE.
Berbary, L. (2015). Creative analytic practices: Onto-epistemological and theoretical attachments,
uses, and constructions within humanist qualitative leisure research. International Leisure
Review, 2(4), 27–55.
Bochner, A. (2018). Unfurling rigor: On continuity and change in qualitative inquiry. Qualitative
Inquiry, 24(6), 359–368. doi:10.1177/1077800417727766
Borneman, J., & Hammoudi, A. (Eds.). (2009). Being there: The fieldwork encounter and the mak-
ing of truth. Berkeley, CA: University of California Press.
Campbell, J., Quincy, C., Osserman, J., & Pedersen, O. (2013). Coding in-depth semistructured
interviews: Problems of unitization and intercoder reliability and agreement. Sociological
Methods & Research, 42(3), 294–320. doi:10.1177/0049124113500475
Caretta, M. A. (2016). Member checking: A feminist participatory analysis of the use of prelimin-
ary results pamphlets in cross-cultural, cross-language research. Qualitative Research, 16(3),
305–318. doi:10.1177/1468794115606495
Caretta, M. A., & Riano, Y. (2016). Feminist participatory methodologies in geography: Creating
spaces of inclusion. Qualitative Research, 16(3), 258–266. doi:10.1177/1468794116629575
Cho, J., & Trent, A. (2006). Validity in qualitative research revisited. Qualitative Research, 6(3),
319–340. doi:10.1177/1468794106065006
Creswell, J. (2013). Qualitative inquiry and research design: Choosing among five approaches (3rd
ed.). Los Angeles, CA: SAGE.
18 J. ROSE AND C. W. JOHNSON

Creswell, J. (2014). Research design: Qualitative, quantitative, and mixed methods approaches (4th
ed.). Los Angeles, CA: SAGE.
Creswell, J., & Miller, D. (2000). Determining validity in qualitative inquiry. Theory into Practice,
39(3), 124–130. doi:10.1207/s15430421tip3903_2
Davis, C., & Lachlan, K. (2017). Straight talk about communication research methods (3rd ed.).
Dubuque, IA: Kendall-Hunt.
Decrop, A. (1999). Triangulation in qualitative tourism research. Tourism Management, 20(1),
157–161. doi:10.1016/S0261-5177(98)00102-2
Denzin, N. (1978). The research act: A theoretical introduction to sociological methods. New York,
NY: McGraw-Hill.
Denzin, N. (2013). The elephant in the living room, or extending the conversation about the pol-
itics of evidence. In N. Denzin & Y. Lincoln (Eds.), Collecting and interpreting qualitative
materials (pp. 517–546). Los Angeles, CA: SAGE.
Denzin, N., & Giardina, M. (2009). Qualitative inquiry and social justice: Toward a politics of
hope. In N. Denzin & M. Giardina (Eds.), Qualitative inquiry and social justice: Toward a pol-
itics of hope (pp. 11–51). Walnut Creek, CA: Left Coast Press.
Dixon-Woods, M., Bonas, S., Booth, A., Jones, D., Miller, T., Sutton, A., … Young, B. (2006).
How can systematic reviews incorporate qualitative research? A critical perspective. Qualitative
Research, 6(1), 27–44. doi:10.1177/1468794106058867
Dowling, R. (2008). Geographies of identity: Labouring in the ‘neoliberal’ university. Progress in
Human Geography, 32(6), 812–820. doi:10.1177/0309132507088032
Dreher, M. (1994). Qualitative research methods from the reviewer’s perspective. In J. Morse
(Ed.), Critical issues in qualitative research methods (pp. 281–297). Thousand Oaks, CA: SAGE.
Dupuis, S. (1999). Naked truths: Towards a reflexive methodology in leisure research. Leisure
Sciences, 21(1), 43–64. doi:10.1080/014904099273282
Ellingson, L. L. (2009). Engaging crystallization in qualitative research. Thousand Oaks, CA:
SAGE.
Farmer, T., Robinson, K., Elliott, S., & Eyles, J. (2006). Developing and implementing a triangula-
tion protocol for qualitative health research. Qualitative Health Research, 16(3), 377–394. doi:
10.1177/1049732305285708
Flyvbjerg, B. (2001). Making social science matter. Cambridge, UK: Cambridge University Press.
Foucault, M. (1970). The order of things: An archaeology of the human sciences. New York, NY:
Vintage.
Furman, R., Lietz, C., & Langer, C. (2006). The research poem in international social work:
Innovations in qualitative methodology. International Journal of Qualitative Methods, 5(3),
24–34. doi:10.1177/160940690600500305
Gibbs, G. R. (2007). Analyzing qualitative data. In U. Flick (Ed.), The SAGE qualitative research
kit. Thousand Oaks, CA: SAGE.
Harrison, J., MacGibbon, L., & Morton, M. (2001). Regimes of trustworthiness in qualitative
research: The rigors of reciprocity. Qualitative Inquiry, 7(3), 323–345. doi:10.1177/
107780040100700305
Harvey, D. (2005). A brief history of neoliberalism. Oxford, UK: Oxford University Press.
Jackson, A. Y., & Mazzei, L. A. (2012). Thinking with theory in qualitative research: Viewing data
across multiple perspectives. New York, NY: Routledge.
Johnson, C., & Parry, D. (2015a). Contextualizing qualitative research for social justice. In C.
Johnson & D. Parry (Eds.), Fostering social justice through qualitative inquiry: A methodological
guide (pp. 11–22). Walnut Creek, CA: Left Coast Press.
Johnson, C., & Parry, D. (2015b). Common features of qualitative inquiry. In C. Johnson & D.
Parry (Eds.), Fostering social justice through qualitative inquiry: A methodological guide (pp.
43–70). Walnut Creek, CA: Left Coast Press.
Kumm, B., & Berbary, L. (2018). Questions for postqualitative inquiry: Conversations to come.
Leisure Sciences, 40(1–2), 71–84. doi:10.1080/01490400.2017.1376014
Kvale, S. (1996). Interviews: An introduction to qualitative research interviewing. Thousand Oaks,
CA: SAGE.
 JOURNAL OF LEISURE RESEARCH 19

Lather, P. (1986a). Research as praxis. Harvard Educational Review, 56(3), 257–277. doi:10.17763/
haer.56.3.bj2h231877069482
Lather, P. (1986b). Issues of validity in openly ideological research: Between a rock and a soft
place. Interchange, 17(4), 63–84. doi:10.1007/BF01807017
Lather, P. (2006). Foucauldian scientificity: Rethinking the nexus of qualitative research and edu-
cational policy analysis. International Journal of Qualitative Studies in Education, 19(6),
783–791. doi:10.1080/09518390600976006
LeCompte, M. D., & Goetz, J. P. (1982). Problems of reliability and validity in ethnographic
research. Review of Educational Research, 51, 31–60. doi:10.3102/00346543052001031
Leech, N., & Onwuegbuzie, A. (2007). An array of qualitative data analysis tools: A call for
data analysis triangulation. School Psychology Quarterly, 22(4), 557–584. doi:10.1037/1045-3830.
22.4.557
Lincoln, Y. (1995). Emerging criteria for quality in qualitative and interpretive research.
Qualitative Inquiry, 1(3), 275–289. doi:10.1177/107780049500100301
Lincoln, Y., & Guba, E. (1985). Naturalistic inquiry. Beverly Hills, CA: SAGE.
Lincoln, Y., Lynham, S., & Guba, E. (2013). Paradigmatic controversies, contradictions, and
emerging confluences, revisited. In N. Denzin & Y. Lincoln (Eds.), The landscape of qualitative
research (4th ed., pp. 199–266). Thousand Oaks, CA: SAGE.
Low, S. (2014). Spatializing culture: An engaged anthropological approach to space and places.
New York, NY: Routledge.
Marshall, C., & Rossman, G. (2016). Designing qualitative research (6th ed.). Los Angeles, CA:
SAGE.
Maxwell, J. (2013). Qualitative research design: An interactive approach (3rd ed.). Los Angeles,
CA: SAGE.
McCoy, K. (2012). Toward a methodology of encounters: Opening to complexity in qualitative
research. Qualitative Inquiry, 18(9), 762–772. doi:10.1177/1077800412453018
Miles, M., Huberman, A. M., & Saldana, J. (2014). Qualitative data analysis: A methods source-
book (3rd ed.). Los Angeles, CA: SAGE.
Miller, M., Grimwood, B., & Arai, S. (2015). Ascetic practices for reflexively navigating power,
privilege and freedoms in leisure research. Leisure/Loisir, 39(3–4), 383–402. doi:10.1080/
14927713.2015.1116204
Morse, J. (2015). Critical analysis of strategies for determining rigor in qualitative inquiry.
Qualitative Health Research, 25(9), 1212–1222. doi:10.1177/1049732315588501
Morse, J., Barrett, M., Mayan, M., Olson, K., & Spiers, J. (2002). Verification strategies for estab-
lishing reliability and validity in qualitative research. International Journal of Qualitative
Methods, 1(2), 13–22. doi:10.1177/160940690200100202
Parry, D., & Johnson, C. (2007). Contextualizing leisure research to encompass complexity in
lived leisure experience: The need for creative analytic practice. Leisure Sciences, 29(2),
119–130.
Parry, D., Johnson, C., & Stewart, W. (2013). Leisure research for social justice: A response to
Henderson. Leisure Sciences, 35(1), 81–87. doi:10.1080/01490400.2013.739906
Patton, M. Q. (2014). Qualitative research and evaluation methods (4th ed.). Los Angeles, CA:
SAGE.
Pink, S., & Morgan, J. (2013). Short-term ethnography: Intense routes to knowing. Symbolic
Interaction, 36(3), 351–361. doi:10.1002/symb.66
Rennie, D., Watson, K., & Monteiro, A. (2002). The rise of qualitative research in psychology.
Canadian Psychology/Psychologie Canadienne, 43(3), 179–189. doi:10.1037/h0086914
Richardson, L. (1997). Fields of play: Constructing an academic life. New Brunswick, NJ: Rutgers
University Press.
Richardson, L., & St. Pierre, E. (2005). Writing: A method of inquiry. In N. Denzin & Y. Lincoln
(Eds.), The SAGE handbook of qualitative research (3rd ed., pp. 959–978). Thousand Oaks, CA:
SAGE.
Roller, M., & Lavrakas, P. (2015). Applied qualitative research design: A total quality framework
approach. New York, NY: Guilford Press.
20 J. ROSE AND C. W. JOHNSON

Rose, J., & Dustin, D. (2009). The neoliberal assault on the public university: The case of recre-
ation, park, and leisure research. Leisure Sciences, 31(4), 397–402. doi:10.1080/
01490400902988333
Rose, J., Harmon, J., & Dunlap, R. (2018). Becoming political: An expanding role for critical leis-
ure studies. Leisure Sciences, 40(7), 649–662. doi:10.1080/01490400.2018.1536569
Rossman, G., Rallis, S., & Kuntz, A. (2010). Validity: Mapping diverse perspectives. In E. Baker,
B. McGaw, & P. Peterson (Eds.), The international encyclopedia of education (3rd ed., pp.
505–513). London, UK: Elsevier.
Saldana, J. (2013). The coding manual for qualitative researchers (2nd ed.). Thousand Oaks, CA:
SAGE.
Schwandt, T. (1996). Farewell to criteriology. Qualitative Inquiry, 2(1), 58–72. doi:10.1177/
107780049600200109
Seale, C. (1999a). The quality of qualitative research. London, UK: SAGE.
Seale, C. (1999b). Quality in qualitative research. Qualitative Inquiry, 5(4), 465–478. doi:10.1177/
107780049900500402
Spall, S. (1998). Peer debriefing in qualitative research: Emerging operational models. Qualitative
Inquiry, 4(2), 280–292. doi:10.1177/107780049800400208
Sparkes, A., & Smith, B. (2009). Judging the quality of qualitative research: Criteriology and rela-
tivism in action. Psychology of Sport and Exercise, 10(5), 491–497. doi:10.1016/j.psychsport.
2009.02.006
Stake, R. E. (1995). The art of case study research. Thousand Oaks, CA: SAGE.
Stuart, F. (2017). Reflexivity: Introspection, positionality, and the self as research instrument –
Toward a model of abductive reflexivity. In C. Jerolmack & S. Khan (Eds.), Approaches to eth-
nography: Analysis and representation in participant observation (pp. 211–238). New York, NY:
Oxford University Press.
Thomas, J. (1993). Doing critical ethnography. Newbury Park, CA: SAGE.
Tracy, S. (2010). Qualitative quality: Eight “big-tent” criteria for excellent qualitative research.
Qualitative Inquiry, 16(10), 837–851. doi:10.1177/1077800410383121
Whittemore, R., Chase, S. K., & Mandle, C. L. (2001). Validity in qualitative research. Qualitative
Health Research, 11, 522–537. doi:10.1177/104973201129119299
Wolcott, H. F. (1990). On seeking – and rejecting – validity in qualitative research. In E. W.
Eisner & A. Peshkin (Eds.), Qualitative inquiry in education: The continuing debate (pp.
121–152). New York, NY: Teachers College Press.
Yin, R. K. (2012). Applications of case study research (3rd ed.). Thousand Oaks, CA: SAGE.