Preventive Veterinary Medicine 228 (2024) 106234

Contents lists available at ScienceDirect

Preventive Veterinary Medicine

journal homepage: www.elsevier.com/locate/prevetmed

Development of a decision support tool to compare diagnostic strategies for

establishing the herd status for infectious diseases: An example with
Salmonella Dublin infection in dairies
Maryse Michèle Um a, b, c, Simon Dufour a, b, c, *, Luc Bergeron d, Marie-Lou Gauthier d, Marie-
Ève Paradis b, e, Jean-Philippe Roy b, f, Myriam Falcon g, Elouise Molgat h, André Ravel a
a
Department of Pathology and Microbiology, Faculty of Veterinary Medicine, Université de Montréal, Saint-Hyacinthe, Canada
b
Op+lait FRQNT Research Group, Faculty of Veterinary Medicine, Université de Montréal, Saint-Hyacinthe, Canada
c
Research Group in Epidemiology of Zoonoses and Public Health, Faculty of Veterinary Medicine, Université de Montréal, Saint-Hyacinthe, Canada
d
Ministère de l’Agriculture, des Pêcheries et de l’Alimentation du Québec, Canada
e
Association des Médecins Vétérinaires Praticiens du Québec, Saint-Hyacinthe, Québec, Canada
f
Department of Clinical Sciences, Faculty of Veterinary Medicine, Université de Montréal, Saint-Hyacinthe, Canada
g
Les Producteurs de lait du Québec, Longueuil, Canada
h
Lactanet, Sainte-Anne-de-Bellevue, Canada

A R T I C L E I N F O A B S T R A C T

Keywords: The diagnosis of infectious diseases at herd level can be challenging as different stakeholders can have conflicting
Multi-criteria decision analysis priorities. The current study proposes a “proof of concept” of an approach that considers a reasonable number of
Diagnostic criteria to rank plausible diagnostic strategies using multi-criteria decision analysis (MCDA) methods. The
Infectious disease
example of Salmonella Dublin diagnostic in Québec dairy herds is presented according to two epidemiological
Dairy herd
Salmonella Dublin
contexts: (i) in herds with no history of S. Dublin infection and absence of clinical signs, (ii) in herds with a
previous history of infection, but absence of clinical signs at the moment of testing. Multiple multiparty ex­
changes were conducted to determine: 1) stakeholders’ groups; 2) the decision problem; 3) solutions to the
problem (options) or diagnostic strategies to be ordered; 4) criteria and indicators; 5) criteria weights; 6) the
construction of a performance matrix for each option; 7) the multi-criteria analyses using the visual preference
ranking organization method for enrichment of evaluations approach; 8) the sensitivity analyses, and 9) the final
decision. A total of nine people from four Québec’s organizations (the dairy producers provincial association
along with the DHI company, the ministry of agriculture, the association of veterinary practitioners, and experts
in epidemiology) composed the MCDA team. The decision problem was “What is the optimal diagnostic strategy
for establishing the status of a dairy herd for S. Dublin infection when there are no clinical signs of infection?”.
Fourteen diagnostic strategies composed of the three following parameters were considered: 1) biological
samples (bulk tank milk or blood from 10 heifers aged over three months); 2) sampling frequencies (one to three
samples collection visits); 3) case definitions to conclude to a positive status using imperfect milk- or blood-ELISA
tests. The top-ranking diagnostic strategy was the same in the two contexts: testing the bulk tank milk and the
blood samples, all samples collected during one visit and the herd being assigned a S. Dublin positive status if one
sample is ELISA-positive. The final decision favored the top-ranking option for both contexts. This MCDA
approach and its application to S. Dublin infection in dairy herds allowed a consensual, rational, and transparent
ranking of feasible diagnostic strategies while taking into account the diagnostic tests accuracy, socio-economic,
logistic, and perception considerations of the key actors in the dairy industry. This promising tool can be applied
to other infectious diseases that lack a well-established diagnostic procedure to define a herd status.

* Corresponding author at: Department of Pathology and Microbiology, Faculty of Veterinary Medicine, Université de Montréal, Saint-Hyacinthe, Canada.
E-mail address: simon.dufour@umontreal.ca (S. Dufour).

https://doi.org/10.1016/j.prevetmed.2024.106234
Received 13 March 2024; Received in revised form 26 April 2024; Accepted 18 May 2024
Available online 24 May 2024
0167-5877/© 2024 The Author(s). Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-
nc-nd/4.0/).
M.M. Um et al.
1. Introduction
Infectious diseases continue to represent a threat to dairy production
in terms of animal health, public health, and economic losses (Ras­
mussen et al., 2021; Drewe et al., 2023). Determining herd-level status
for certain diseases plays a key role in deciding the prevention and
control measures to put in place (e.g. on the occasion of animal trade,
herd management, organization of pasteurization circuits, biosecurity
measures, etc.); it relies on effective diagnostic approaches. However,
for some diseases, diagnostic challenges persist even where efforts for
implementing surveillance and control programs have been put in place
for decades (Nielsen et al., 2021; Nobrega et al., 2023). These challenges
are often associated with the availability of only imperfect tests (anti­
bodies-, culture-, PCR-based approaches on aggregated or individual
samples), the chronic nature of some diseases and absence of clear
clinical signs, the pathogenesis of the agent involved and the difficulty to
sometimes define the “carrier state” of an infected individual. These
difficulties have considerable consequences (direct costs to the pro­
ducers, impacts due to misclassification errors like “culling false-positive
animals”, compromised trades, disruption in milk collection associated
with false-positive results; or risky trades, herds escaping authorities’
interventions due to false-negative results).
The implementation of a diagnostic strategy aiming at establishing
the herd-level status for an infectious disease requires an optimal se­
lection of: the age of tested animals, the number of animals tested, the
optimal timing with regards to the infection curve, the type of biological
samples collected, the tests used, and the epidemiological contexts of the
disease (Um et al., 2022). Salmonella Dublin infection in dairy cattle
exemplifies all the latter challenges. In young animals, the infection
often causes non-specific enteric signs, respiratory signs, and sudden
death due to septicemia. In adults, it can lead to abortion. In many herds,
however, the disease is mostly subclinical. The bacteria is host-adapted,
and thus, generates subclinical carriers among cows, making their
identification more complicated for veterinary practitioners and pro­
ducers (Nielsen, 2013).
In the Québec province, Canada, the S. Dublin emerged in 2011 in
dairy cattle and continues to spread in dairy herds (RAIZO, 2022). The
diagnosis of S. Dublin can be carried out by demonstrating the presence
of the bacteria in clinical or necropsy samples by culture or PCR or, in
the absence of clinical signs in the herd, with the measurement of an­
tibodies for S. Dublin using an ELISA test performed on bulk tank milk
and/or individual animals blood samples (MAPAQ, 2023), as it is done
in other countries where the disease is endemic (Agren et al., 2018;
Holschbach and Peek, 2018). However, both ELISA tests (i.e. on milk
and blood) are imperfect (Um et al., 2022). It appears that all the
aforementioned considerations to select a diagnostic strategy could
represent divergent issues for the potential parties dealing with the
disease (i.e. economic losses for dairy producers, false positive and/or
false negative results for animal health authorities, etc.). For example,
for dairy producers, the preference could go to one bulk tank milk
testing every six months as it is a less costly, and time-saving approach.
While, for animal health authorities, the preference could be a combi­
nation of blood testing from non-milking animals (heifers) and bulk tank
milk testing with a requirement that at least one test should be positive
to consider the herd S. Dublin-positive, even though, this strategy could
be more expensive and logistically more demanding than the latter.
To aid decision-makers in adopting a compromise for the diagnostic
strategy to be implemented, methods such as multi-criteria decision
analysis (MCDA) can be used. It considers various and sometimes con­
flicting perspectives. It originates from the field of operational research
(Keeney and Raiffa, 1976) and allows comparing and ranking potential
solutions to a problem without an obvious solution according to several
defined criteria. Its advantages are that: (i) stakeholders from all sectors
with different ideas and preferences about a solution can present their
views, which brings transparency in decision-making; (ii) measurable
criteria are used to compare solutions favoring reliability; (iii)
2
Preventive Veterinary Medicine 228 (2024) 106234
qualitative and quantitative indicators are used for criteria along with
mathematical equations, which improves rationality; (iv) all judgements
of stakeholders are emphasized through weights that are attributed to
each criterion; (v) mathematical optimization, algorithms and advanced
scientific computing are used to rank the potential solutions (Dodgson
et al., 2009). The MCDA methods have been widely used in management
science and various areas of engineering (Wiecek et al., 2008). Their
application in the field of animal or public health has been increasing
over the last decade, notably for ranking pathogens or diseases in certain
contexts (Cox et al., 2013; Hongoh et al., 2017) or for identifying and
assessing options for surveillance and control programs (e.g. Lyme dis­
ease, foot-and-mouth disease, bovine tuberculosis) (Aenishaenslin et al.,
2013; Corbellini et al., 2020). Its application to compare diagnostic tests
in human medicine or potential diagnostic strategies for establishing the
herd status for infectious diseases is more recent (Sayan et al., 2020;
Guetin-Poirier et al., 2022). The MCDA methods allow the conception of
frameworks that are reusable and updatable according to the
decision-making context. There is clearly a need for such reproducible
and transparent decision-making process in the field of diagnostic of
animal infectious diseases.
The objectives of the study were: 1) to develop a MCDA tool that
could be used to compare diagnostic strategies aiming at establishing the
herd status for infectious diseases in dairies and in different epidemio­
logical contexts (e.g. herds with or without history of known previous
infection); and 2) to compare, using that latter tool, diagnostic strategies
for establishing the herd status for S. Dublin infection in the two
different contexts.
2. Materials and methods
This study, hereafter named the MCDA project, was conducted be­
tween June 2021 and April 2023. The study was part of a larger project
entitled “Validation of bulk tank milk sampling strategies and identifi­
cation of risk factors for Salmonella Dublin in dairies” hereafter called S.
Dublin project with the overall objectives of suggesting a diagnostic
strategy for determination of the herd-level S. Dublin status in dairies
and identifying farm-level risk factors for S. Dublin infections (Um et al.,
2020, 2022, 2023). The current part of that larger study received the
approval of the Université de Montréal’s Science and Health Research
Ethics Committee (or Comité d’éthique de la recherche en science et en
santé de l’Université de Montréal), project number 2022–1629.
MCDA actually encompasses a variety of methods with different
approach and technical features (Greco et al., 2016). For this project, we
chose the preference ranking organization method for enrichment of
evaluations (PROMETHEE) because of its simplicity in conception, its
clearness, and its user-friendly features (Brans et al., 1986; Behzadian
et al., 2010). This outranking method consists in pairwise comparisons
of the options to solve a problem according to evaluation criteria and
preferences of the stakeholders, which is more intuitive than defining
“what is good” and “what is bad” (Mareschal, 2013). We followed and
adapted the steps proposed to guide the MCDA process (Aenishaenslin
et al., 2019). Table 1 lists these steps whereas the following sections
detail each of them.
2.1. Identifying the participating stakeholders
The initial concern of the S. Dublin project participants that led to
undertake the MCDA was about recommending a diagnostic strategy for
establishing the status of a dairy herd for S. Dublin infection when there
are no clinical signs of infection and in a context where no nationwide
surveillance for such infection had been planned. The goal was to help
the veterinarians in using a common and optimal strategy when their
dairy clients want to know their herd’s status for this infection.
To identify the relevant stakeholders, the MCDA project coordinator
(i.e. the first author) invited all S. Dublin project participants to a
meeting. During that meeting, they used the rainbow diagram tool
M.M. Um et al.
Table 1
Steps for a multi-criteria decision analysis and the main outputs of its application
for recommending a diagnostic strategy in Québec, Canada for establishing the
status of a dairy herd for Salmonella Dublin (S. Dublin) infection when there are
no clinical signs of infection.
Steps Outputs
1. Identifying the participating Four organizations:
stakeholders 1) the dairy producers of Québec along with
the private dairy herd improvement
company in charge of individual and bulk
milk analyses and diagnostic in dairy cattle
sector (PLQ);
2) the ministry of agriculture. fisheries and
food of Québec (MAPAQ);
3) the association of veterinary practitioners
of Québec (AMVPQ);
4) epidemiologists from the veterinary faculty
(FMV)
With 2 representatives for each organization +
1 coordinator
2. Refining the decision problem One general question: what is the optimal
diagnostic strategy for establishing the status
of a dairy herd for S. Dublin infection when
there are no clinical signs of infection?
Two contexts:
1) herds with no previous history of S. Dublin
infection
2) herds with history of S. Dublin infection
3. Defining the options for the 14 diagnostic strategies (Fig. 1)
diagnostic strategy
4. Defining the criteria and their 10 criteria and their indicator (Table 2 and
indicators Table S1)
5. Weighting the criteria The criteria weights determined by each
participating stakeholder and those reached by
consensus (Table S2)
6. Assessing the performance of The performance matrix (Table 3)
each option on each criterion The preference functions (Table S3)
7. Multicriteria analysis based on The option profiles (Fig. 2)
PROMETHEE The option net outranking flow and their
ranking by participating stakeholders (Table 4)
Map of the relative positioning of criteria and
options (GAIA) (Fig. 3)
Map of the relative positioning of participating
stakeholders and the options (Fig. 4)
8. Sensitivity analysis No important differences within and between
the participating stakeholders were identified (
Figs. 3 and 4).
9. Decision on the optimal Two diagnostic strategies recommended and
diagnostic strategy another one found interesting under certain
circumstances (see text).
(Chevalier and Buckles, 2008) for the collective identification of the
relevant stakeholders to participate into the MCDA. Stakeholders were
defined as key collective actors (i.e. not individual but organization or
group of actors sharing the same interest and stake) who could be
affected by the decision (e.g. those applying the diagnostic strategy or
dealing with the herd status resulting from the diagnostic strategy, or
who could influence its development). The meeting participants iden­
tified all potential stakeholders and used the rainbow diagram to char­
acterize each along two dimensions: the most, the moderately or the
least affected stakeholders, and the most or the least influential stake­
holders. The coordinator then invited all identified organizations as the
most influential and either the most or moderately affected stakeholders
to participate into the MCDA project. The participating stakeholders
plus the MCDA coordinator are referred as to the MCDA team. The
MCDA team then participated to each of the next steps by holding
meetings and sharing documents.
2.2. Refining the decision problem
The MCDA team reflected on the problem as initially stated, ac­
cording to the context of S. Dublin dairy herd infections in Québec, in
3
Preventive Veterinary Medicine 228 (2024) 106234
order to come up by consensus with a more specific statement of the
decision problem that was at stake. More specifically, the context
included the epidemiology of the disease in Québec dairy herds as
known from the surveillance of clinical cases and the findings of the
larger S. Dublin project. Of interest, those sources of information esti­
mated the herd-level S. Dublin prevalence at 6.8% in a convenience
sample of dairy herds in Québec, but at 25% in herds with a previous
history of this latter infection (Um et al., 2022), suggesting at least two
epidemiological contexts for the disease in Québec.
2.3. Defining the options
The MCDA team brainstormed on the potential diagnostic strategies
that could be used in these two contexts for establishing the herd status
for S. Dublin infection in the absence of clinical signs. The discussion
was informed by practical aspects, by the findings of the ongoing S.
Dublin project (Um et al., 2020, 2022, 2023), the availability of labo­
ratory tests in Québec, the testing procedures already used by the pro­
vincial animal health authorities in case of clinical signs of S. Dublin
infection in dairy cattle (MAPAQ, 2023), and the S. Dublin surveillance
programs in place in northern European countries that have been
dealing with S. Dublin infection for decades (Jordan et al., 2008; Niel­
sen, 2013; Agren et al., 2018). Full consensus was the rule for deciding
on the diagnostic strategies to include into the MCDA.
2.4. Defining criteria and indicators
The MCDA team brainstormed on the main issues and concerns they
had about the future diagnostic strategy, i.e. what could bother the
stakeholders, what feature the strategy should have or avoid for facili­
tating the implementation and the usefulness of the strategy. For
example, cost, delay, and diagnostic precision are usual concerns when
selecting a diagnostic strategy. Each MCDA team member had the op­
portunity to express its concerns and perceptions on the features that the
diagnostic strategy should fulfill. Through discussion, a set of common
and shared general issues and concerns was defined by full consensus.
Then the MCDA team further defined these general issues and concerns
into a number of potential criteria (usually a few criteria per issue) that
would be the basis for comparing the diagnostic strategies. All proposed
criteria were explained, discussed, and appraised according to three
rules: 1) each criterion must be relevant to the decision problem, which
was comparing diagnostic strategies; 2) each criterion must be associ­
ated with a measurable (quantitatively or qualitatively) characteristic of
the diagnostic strategies; and 3) the value for the criterion should differ
for at least two diagnostic strategies. Every general issue or concern
needed to have at least one criterion, and the team aimed at limiting the
total number of criteria to 10–15. The decision about the final list of
general issues or concerns and the final list of criteria had to be
consensual.
The MCDA team then defined the indicator (i.e. how the criterion is
measured precisely) for each criterion and determined its measurement
scale, which had to be specific, measurable, applicable, realistic, and
timely. Full consensus was, again, used to finalize all indicators and
scales. The MCDA team also determined whether the value of each in­
dicator should be maximized or minimized (i.e. the desirable outcome
for the indicator) in the subsequent MCDA analysis.
2.5. Weighting the criteria
The fixed-point allocation method was used to quantify the criteria
weights according to a two-step hierarchical process (Aenishaenslin
et al., 2019). It was applied for each context separately. First, each pair
of representatives of a given participating stakeholder distributed 100
points over the general issues or concerns identified. Second, they
distributed 100 points over the criteria included within each issue. The
final criterion weight for each stakeholder was the multiplication of each
M.M. Um et al.
criterion weight within the issue by the corresponding issue weight.
Then, the whole MCDA team met, evaluated and compared the different
stakeholder’s weighting proposal, and reached a consensus on an overall
weighting scheme. The subjective weighting method was selected
among the various weighting methods available because of its easiness
to implement (Nemeth et al., 2019) and because it provides similar re­
sults to more objective or more sophisticated methods (van Til et al.,
2014).
2.6. Assessing the performance of each option on each criterion
The performance was assessed diversely depending on the indicators.
The value for each option was directly computed for some indicators,
computed based on the findings of the S. Dublin project for others, or
appraised through specifically designed research tools (e.g. survey). This
step yielded all the information needed for the performance matrix, i.e. a
grid summarizing the value of each diagnostic strategy on each criterion.
2.7. Multi-criteria analyses
Basically, the multi-criteria analysis with PROMETHEE first consists
in comparing each pair of diagnostic strategies on all criteria yielding
unicriterion scores based on the values in the performance matrix. Those
pairwise comparisons require the definition of the preference function
for each criterion. The preference function determines the extent of the
preference of one diagnostic strategy over another by setting the mini­
mal difference between the two values to state that one diagnostic
strategy is preferred over another, the shape of the relation between the
difference and the degree of outranking, and the maximal difference
between two values for considering that the maximum degree of out­
ranking is reached. A unicriterion score can be positive, indicating that
overall, the diagnostic strategy outranked most of the others; it can be
negative, indicating that, overall, the diagnostic strategy is outranked by
most of the others. Second, the analysis aggregates all unicriterion scores
of a given diagnostic strategy into an overall net outranking flow using
the criteria weights. The diagnostic strategies can then be ranked ac­
cording to their net outranking flow values. More details of the com­
putations behind the analysis have been well described (Brans and De
Smet, 2016). We used Visual PROMETHEE (software package Academic
version 1.9) (Mareschal, 2013) to perform all analyses. In addition, this
software allows displaying the analysis outputs in a variety of tabular
and graphical formats, which are helpful to interpret the results mean­
ingfully. One interesting tool is the Geometrical Analysis for Interactive
Aid (GAIA). It allows, among other things, projecting the options and the
criteria values after a principal component analysis of the performance
values, yielding a bi or tri dimensional plane with the greatest variance,
and is helpful to explore the proximity between diagnostic strategies, the
independence between the criteria, and the relationship between the
diagnostic strategies and the criteria. In addition, GAIA allows the vector
of the weighted criteria (referred as to the decision axis) to be shown on
the same plane, thus highlighting toward which diagnostic strategy the
decision points out. Note that, except for the decision axis, the GAIA
plane is the same for any stakeholder. Finally, with GAIA, the net out­
ranking flows of the various stakeholders can be aggregated and dis­
played on another plane showing the extent of divergence between
stakeholders on their ranking of the options under evaluation. To make
our results reproducible, the completed Visual PROMETHEE files (n=2;
one per context) are available at https://doi.org/10.5683/S
P3/87HRGH.
2.8. Sensitivity analyses
Since the preference flows are linear functions of the weights of
criteria, sensitivity analyses on the criteria’s weights are critical.
Focusing on the top five diagnostic strategies, we modified the weight of
one criterion at a time (by adding -5–5 points to its value) and visually
4
Preventive Veterinary Medicine 228 (2024) 106234
detected whether the ranking of the five top options changed using the
Walking Weights tool in Visual PROMETHEE. In addition, we assessed
whether the range of weights of an option, within which the five top
outranking options stayed the same, was below 4 points using the Sta­
bility Intervals tools. The threshold of 4 points was set by considering a
20% uncertainty in the weight value for a uniform distribution of weight
(i.e. - and +20% of the 100 total allocated points/10 criteria means a
range of 8–12 points). All this was undertaken for each criterion and for
each participating stakeholder plus the consensual weighting. Finally,
we used the GAIA Decision-Maker-Brain for each stakeholder partici­
pant to assess the sensitivity of its decision axis to the criterias’ weights
(Brans and De Smet, 2016). This tool automatically computes the deci­
sion axis based on a range of weights instead of a single value and
projects them as an ellipse on the GAIA plane. The outranking is robust
when the ellipse does not include the origin of the plane. The range for
the varying weights was set at 25% of its value (e.g. if the weight is 8%,
the range was set to vary between 6% and 10%).
2.9. Final decision
Informed by the numerous outputs of the results, the MCDA team
met and discussed about the optimal diagnostic strategy to recommend
and decide on the optimal diagnostic strategy, again by full consensus.
3. Results
3.1. Problem, stakeholders, criteria, performance matrix
3.1.1. The participating stakeholders
Results from the stakeholders’ identification exercise are provided as
supplementary materials (https://doi.org/10.5683/SP3/87HRGH).
Initially, the rainbow diagram led to consider 18 different stakeholders.
Four were identified as “the most influent and the most affected”: the
dairy producers of Québec or les Producteurs de Lait du Québec - PLQ;
the beef producers of Québec or les Producteurs de Bovins du Québec -
PBQ; the ministry of agriculture, fisheries and food of Québec or Min­
istère de l’Agriculture, des Pêcheries et de l’Alimentation du Québec -
MAPAQ; the association of veterinary practitioners of Québec or Asso­
ciation des Médecins Vétérinaires Praticiens du Québec - AMVPQ. Four
were identified as “the most influent, but moderately affected”: the
private DHI company in charge of individual and bulk milk analyses and
diagnostics in dairy cattle sector or Lactanet; epidemiologists from the
veterinary faculty of the Université de Montréal or Faculté de Médecine
Vétérinaire - FMV; the Québec dairy breeds council or Conseil Québécois
des Races Laitières - CQRL; and the cattle auctions of the province of
Québec. After further discussions, the dairy producers of Québec and the
DHI company were considered as stemming on the same group of in­
terest, that of the dairy industry, and were kept together under the same
stakeholder’s name PLQ. All those identified organizations were offi­
cially invited to be part of the MCDA project. Three declined the invi­
tation: PBQ, CQRL, and the cattle auctions. Therefore, eight
representatives from four organizations plus the MCDA project coordi­
nator were included as part of the MCDA team (Table 1).
3.1.2. The decision problem refined
The decision problem was refined as “What is the optimal diagnostic
strategy for establishing the status of a dairy herd for S. Dublin infection
when there are no clinical signs of infection?”. The term diagnostic
strategy represented the combination of sampling strategies (i.e. bio­
logical material collected, number of tests, number of animals, animals’
categories, frequency of sampling) and the tests to be used (i.e. type,
interpretation, thresholds, case definition). In addition, the MCDA team
emphasized two different contexts for which the decision problem had
to be worked out (Table 1): the “No S. Dublin history” context (i.e. for
herds without a known history of S. Dublin infection), and the “With S.
Dublin history” context (i.e. for herds with a previously confirmed
M.M. Um et al.
history of infection, but still in absence of obvious clinical signs at the
moment of testing).
3.1.3. The options included
The options for the diagnostic strategies were composed of three key
elements: 1) the biological sample (i.e. either bulk tank milk or blood
from 10 heifers aged over three months); 2) the sampling frequency (i.e.
from one to three samples collection visits); 3) the case definition
required to classify a herd as S. Dublin-positive (i.e. the number of test-
positive bulk tank samples and/or seropositive heifers to consider a herd
as positive). Only one type of laboratory test was considered for both
milk and blood samples: the ELISA test (PrioCHECK™ Salmonella Ab
Bovine Dublin) with the same cut-off set at PP% ≥ 35. The MCDA team
identified 14 options for the diagnostic strategies (Fig. 1). They were the
same for both disease contexts. For strategies relying on multiple visits,
the time intervals between two successive sampling visits were set to be
one month for strategies involving only bulk milk samples or six months
whenever two blood samples visits were encompassed.
3.1.4. The criteria proposed and their weighting
The MCDA team agreed upon 10 criteria grouped into four general
issues or concerns. The criteria were similar for the two contexts
(Table 2). The identified issues were: diagnostic considerations (four
criteria), socio-economics considerations (two criteria), logistics
Fig. 1. The 14 diagnostic strategies evaluated in a MCDA analysis for establishing the dairy herd status for Salmonella Dublin infection.
5
Preventive Veterinary Medicine 228 (2024) 106234
considerations (two criteria), and perception of producers and veteri­
narians (two criteria). Their corresponding indicators, measurement
scales, direction relative to the decision (maximization or minimiza­
tion), and computation methods or source are presented in supple­
mentary Table S1. Eight indicators were measurable on a quantitative
continuous scale, while two were measurable on a categorical ordinal
scale.
The weight for the criteria varied between the participant stake­
holders and differed between the two disease-diagnostic contexts (sup­
plementary Table S2). Overall, most of the weight was attributed to the
diagnostic considerations (range: 30–55%) and least to the logistic
consideration (range: 5–25%). At the criterion level, the following
criteria received the largest consensual weight when the diagnosis
strategies were applied to herds without a previous history of S. Dublin:
impacts of diagnostic error (15%) and value of a negative diagnostic
(13.5%). When the diagnostic strategies were applied to herds with a
previous history of S. Dublin, then the largest consensual weights were
obtained for impacts of diagnostic error (17.5%) and the representa­
tivity of the sampling (12.5%).
3.1.5. The performance of each diagnostic strategy
The value of each diagnosis strategy for each criterion is displayed in
the performance matrix (Table 3). Some performances obviously were
identical for some criteria between the diagnosis strategies that shared a
M.M. Um et al. Preventive Veterinary Medicine 228 (2024) 106234

Table 2
Concerns, criteria, and indicators proposed for a multicriteria analysis in order to compare diagnostic strategies and ultimately establish the dairy herd status for an
infectious disease (Salmonella Dublin in this case).
Issue or concern Criteria (code) Definition Indicator

Diagnostic Value of a positive diagnostic Probability that a positive result is correct Positive predictive value (PPV)
considerations (1_VD+)
Value of a negative diagnostic Probability that a negative result is correct Negative predictive value (NPV)
(2_VD-)
Credibility of predictive values Level of certainty in the two preceding parameters Sum of the width of the 95% credibility interval of the PPV and
(3_Credibility) NPV
Representativity of the sampling The extent to which the sampling strategy A categorical variable defined by the biological samples collected
(4_Representativity) represents different groups of animals in the herd (milk vs. blood vs. milk and blood) and the number of sampling
vs. a fraction of the animals visits (1 vs. 2 vs. 3 visits)
Socio-economic Operational costs The costs associated with the sampling strategy Costs in $ for producer time, veterinarian fees, and laboratory
considerations (5_Costs) analyses
Impact of diagnostic errors Societal and economic consequences associated Misclassification cost terms according to the epidemiological
(6_Impacts) with diagnostic errors context (Um et al., 2023)
Logistic Delay before final results Time needed to obtain the diagnostic Number of days before obtaining the diagnostic
considerations (7_Delay)
Mobilization of human resources Human resources needed in the field to collect A categorical variable representing the human resources
(8_Resources) samples involved (producers vs. veterinarians vs. both)
Perception Level of producers’ acceptability The extent to which the veterinarian will get the Veterinarian’s perception regarding the acceptability of the
(9_Acceptability) buy-in from the producers regarding the diagnostic diagnostic strategy by producers
strategy
Level of simplicity of results’ Ease for the veterinarian to determine and Acceptability by veterinarians of the diagnostic strategy’s case
interpretation by veterinarians communicate results to the producer definition
(10_Simplicity)

diagnostic strategy feature (e.g. the number of visits, the type of sample
tested). Briefly, the value of a positive diagnostic varied between 28.1%
and 92.6% (mean 49.4%) for herds with no previous S. Dublin infection
history and between 71.2% and 98.7% (mean 84.5%) for herds with a
previous S. Dublin history. Similarly, the value of a negative diagnostic
ranged between 95.4% and 99.3% (mean 96.9%) for herds with no
previous S. Dublin infection history and between 76.5% and 95.5%
(mean 83.5%) for herds with a previous S. Dublin history. The credi­
bility of predictive values criteria varied from 31.2% to 94.3% (mean
67.5%) for herds with no previous S. Dublin infection history and from
22.9% to 87.6% (mean 56.1%) for herds with a previous S. Dublin his­
tory. Five scores (from very poor to very good) were possible for the
representativity of the sampling criteria, and three scores (low, moder­
ate, and high) were possible for the mobilization of human resources
criteria. Seven different values were obtained for the operational costs
criteria (13.50, 27.00, 40.50, 297.90, 311.40, 598.80, and 622.80 CAD),
while four different values were obtained for the delay before final re­
sults criteria (7, 37, 67, and 187 days). Impact of diagnostic errors
ranged from 0.031 to 0.101 (mean 0.057) for herds with no previous S.
Dublin infection history and from 0.15 to 1.147 (mean 0.729) for herds
with a previous S. Dublin history. Level of producers’ acceptability
varied between -0.9 and 4.9 (mean 2.3) for herds with no previous S.
Dublin infection history and between 0.0 and 6.0 (mean 3.3) for herds
with a previous S. Dublin history. Level of simplicity of results’ inter­
pretation by veterinarians ranged from -0.9–6.2 (mean 3.3) for herds
with no previous S. Dublin infection history while it ranged from
-2.5–6.0 (mean 1.8) for herds with a previous S. Dublin history.
maximal for the strategies based on milk and blood samples (M+B).
Conversely, strategies based on one or two milk sample collection visits
performed very well on the other three issues (i.e. socio-economic
consideration, logistic consideration, and perception) compared to
strategies based on milk and blood samples. After taking into consid­
eration the criteria weight and the relative performance of each option,
four options clearly ranked amongst the top options for almost all
participating stakeholders: 1(M+B):1or1, 1(M+B):1or2, 3 M:1, and
1 M:1 by decreasing order of their net outranking flows (Table 4).
The GAIA plane has a good representativity (76% on the first two
axes) and was clearly split into two groups (Fig. 3A). The strategies
based on milk samples were spread out in the left part, whereas the other
(blood samples or combinations of bulk milk and blood samples) were
on the right part with the exception of option 1(M+B):1or2. Three
criteria relative to the diagnostic issue (value of a negative diagnostic,
credibility of predictive values, and representativity of the sampling)
were clustered together and were in opposition with the impact of
diagnostic errors and mobilization of human resources criteria. The two
perception criteria (level of producers’ acceptability and level of
simplicity of results’ interpretation by veterinarians) were clustered
together on the GAIA plane and they pointed, with the delay before final
results criterion, in the opposite direction as compared to the value of a
positive diagnostic criterion.
According to the sensitivity analysis, the ranking of the five top
outranking options was very robust to changes in the weights stake­
holders attributed to the criteria. There were no major divergent views
between the participating stakeholders on the best options (Fig. 4A).

3.2. The multi-criteria analyses
3.2.1. The preference functions
The preference functions used were linear or V-shaped with small
preference thresholds whereas the indifference thresholds were set to
cover the full range of the criterion values (Supplementary Table S3).
3.2.2. Results when testing herds with no previous S. Dublin infection
history
According to the pairwise comparisons, the performances of the 14
diagnostic strategies on the 10 criteria varied considerably (Fig. 2).
Worth noting, the performances related to the diagnostics issue were
relatively minimal for the strategies based on bulk milk samples (M) and
3.2.3. Results when testing herds with a previous S. Dublin infection history
The performances of the strategies for this context were slightly
different compared to those for herds with no previous S. Dublin
infection history (Fig. 2B). Taking into account the weight, the same
strategies ranked amongst the top four options for all participating
stakeholders: 1(M+B):1or1, 2(M+B):(1or1)OR(1or1), 1(M+B):1or2,
and 1B:1 (Table 4). The GAIA map has a good representativity (75% on
the first two axes). It suggested that strategies based on milk testing only
tend to cluster together and were aligned with lower costs and less
resource (Fig. 3B). Strategies based on two visits of blood testing were
aligned with greater delay and lower acceptability, and possibly lower
simplicity. The four criteria related to the diagnostic consideration and
the criterion impact of diagnostic errors were almost aligned and in the

6
M.M. Um et al. Preventive Veterinary Medicine 228 (2024) 106234

Table 3
Performance matrix comparing the 14 options against the 10 criteria for the two contexts of interest (herd with or without history of known previous infection by
Salmonella Dublin).
Diagnostic strategiesa Context Diagnostic considerations Socio-economic Logistic considerations Perception
considerations

1_VD+ 2_VD- 3_Credi- 4_Repre- 5_Costs 6_Im- 7_De- 8_Resources 9_Accep- 10_Sim-
bility sentati- pacts lay tability plicity
vity

% % %Points Score CAD Score Days Score Score Score

1 M:1 No History 46.1 96.1 74.1 Very poor 13.5 0.052 7 Low 4.9 4.4
With 84.4 79.4 46.2 Very poor 13.5 0.969 7 Low 6.0 4.3
History
2 M:1 No History 34.4 96.4 86.2 Poor 27 0.064 37 Low 4.9 3.8
With 76.9 80.9 87.6 Poor 27 0.881 37 Low 5.8 4.0
History
2 M:2 No History 50.8 95.6 92.0 Poor 27 0.050 37 Low 3.3 3.8
With 86.7 77.4 76.2 Poor 27 1.095 37 Low 5.0 0.3
History
3 M:1 No History 38.1 96.8 82.8 Moderate 40.5 0.062 67 Low 3.8 6.2
With 79.6 82.5 77.5 Moderate 40.5 0.792 67 Low 5.5 5.3
History
3 M:3 No History 40.9 95.4 93.3 Moderate 40.5 0.052 67 Low -0.9 -0.9
With 81.4 76.5 85.6 Moderate 40.5 1.147 67 Low 3.0 -2.5
History
1B:1 No History 49.2 97.4 75.2 Very poor 297.9 0.051 7 Moderate 1.3 4.0
With 86 85.5 53.2 Very poor 297.9 0.150 7 Moderate 3.0 1.3
History
1B:2 No History 77.4 96.1 66.3 Very poor 297.9 0.042 7 Moderate 2.7 3.8
With 95.6 79.7 31.2 Very poor 297.9 0.954 7 Moderate 4.5 2.3
History
2B:1 or 1 No History 28.1 98.1 55.9 Poor 595.8 0.101 187 Moderate 0.7 3.1
With 71.2 88.9 75.8 Poor 595.8 0.495 187 Moderate 0.0 1.8
History
2B:2 and 2 No History 49.9 95.5 94.3 Poor 595.8 0.050 187 Moderate -0.9 1.3
With 86.3 76.9 84.2 Poor 595.8 1.121 187 Moderate 0.5 -0.3
History
1(M+B):1 or 1 No History 44.8 98.1 44.6 Good 311.4 0.057 7 High 3.8 4.7
With 83.7 88.8 38.4 Good 311.4 0.482 7 High 4.0 3.5
History
1(M+B):1 or 2 No History 54.6 97.0 56.7 Good 311.4 0.046 7 High 3.8 4.2
With 88.4 83.8 33.0 Good 311.4 0.726 7 High 5.3 3.0
History
2(M+B):(1 or 1) No History 35.7 99.3 36.8 Very good 622.8 0.085 187 High 3.1 3.6
OR(1 or 1) With 77.9 95.5 36.3 Very good 622.8 0.224 187 High 0.8 3.0
History
2(M+B):(1 or 1) AND(1 No History 92.6 97.0 31.2 Very good 622.8 0.031 187 High -0.4 1.1
or 1) With 98.7 83.5 22.9 Very good 622.8 0.741 187 High 1.0 -1.8
History
2(M+B):(1 or 2) No History 48.6 98.3 55.8 Very good 622.8 0.052 187 High 1.8 3.3
OR(1 or 2) With 85.7 89.9 37.5 Very good 622.8 0.433 187 High 2.0 0.9
History
a
The code used for the diagnostic strategies is: the number and the letter prior to the colon give the number of sample collection visits and the type of sample (blood
of 10 young animals (B), bulk milk (M) or both sample types (M+B)), respectively; the number of tests that has to be positive to attribute a positive status to the herd
depending on the diagnostic strategy is after the colon (i.e. 1 bulk milk tests and/or 1 or 2 seropositive animals out of 10 tested per visit).

opposite direction of the strategies based on milk testing only. The
ranking of the options by the participating stakeholders was very robust
to changes in the weights they attributed to the criteria and there were
no major divergent views between the participating stakeholders on the
best strategies (Fig. 4B).
3.3. Final decisions
When testing herds with no previous S. Dublin infection history, the
MCDA team concluded that the most preferable diagnostic strategy was
1(M+B):1or1 (i.e. testing the bulk tank milk and the blood of 10 heifers
>3-month-old, all samples taken during a single farm visit, and the herd
being assigned a positive status whenever the milk sample or one of the
blood samples is positive to the PrioCHECK™ ELISA test). The MCDA
team further made the caution that this diagnostic strategy should be
recommended as a “screening test” to rule out the presence of S. Dublin.
In a herd with no history of S. Dublin and using this diagnostic strategy
in the absence of clinical signs of the disease, we can expect that one out
of two positive results will be a false positive (positive predictive value
of 44.8%; Um et al., 2023).
For testing herds with a previous S. Dublin history, the MCDA team
concluded that the diagnostic strategy 1(M+B):1or1 was the most
relevant. However, the MCDA team also concluded that the diagnostic
strategy 2(M+B):(1or1)OR(1or1) (an approach similar to the previous
one but requiring a repetition of the tests with the same samples
collected during a second visit carried out 6 months later, with a case
definition of 1 positive test to define a herd status as positive) could be
useful under certain conditions. The main advantage of the latter
strategy was the greater confidence in the result for herds testing
negative. In the latter case, when testing negative, only 1 out of 20 herds
(negative predictive value of 95.5%) would possibly be a false negative.
As a comparison, with the 1(M+B):1or1 strategy, among the herds with
a negative result, 2 out of 20 would be expected to be false negative
results. The 2(M+B):(1or1)OR(1or1) strategy could, therefore, be

7
M.M. Um et al. Preventive Veterinary Medicine 228 (2024) 106234

Fig. 2. Stacked unicriterion scores of the 14 diagnostic strategy options assessed on the 10 criteria proposed resulting of all pairwise comparisons of the strategies
without weight or preference functions. A/ Diagnostic strategies for herds with no previous history of Salmonella Dublin infection. B/ Diagnostic strategies for herds
with a previous history of Salmonella Dublin infection. The code for the diagnostic strategies is as followed: the number and letter prior to the colon give the number
of visits and the type of sample (i.e. blood of 10 young animals (B), bulk milk (M) or both sample types (M+B)), respectively. The number of tests that must be
positive to attribute a positive status to the herd depending on the diagnostic strategy is after the colon (i.e. 1 bulk milk test, and/or 1 or 2 seropositive animals out of
10 tested per visit).

8
M.M. Um et al. Preventive Veterinary Medicine 228 (2024) 106234

Table 4
Rank and net outranking flow of the 14 diagnostic strategy options for the two contexts (herds with or without history of known previous infection by Salmonella
Dublin). The strategies are ordered according to their rank obtained with the consensus-built weights.
FMV MAPAQ PLQ AMVPQ Consensus

Diagnostic strategiesa Rank Flow Rank Flow Rank Flow Rank Flow Rank Flow

Herds with no previous S. Dublin history

1(M+B):1or1 1 0.161 1 0.204 1 0.140 2 0.173 1 0.177
1(M+B):1or2 4 0.136 3 0.124 4 0.110 5 0.123 2 0.111
3 M:1 3 0.137 5 0.068 2 0.131 3 0.148 3 0.081
1 M:1 2 0.137 8 0.007 3 0.129 1 0.177 4 0.078
2 M:1 5 0.108 9 -0.016 6 0.088 4 0.135 5 0.047
1B:2 7 0.065 7 0.031 5 0.093 7 0.081 6 0.045
2(M+B):(1or1)OR(1or1) 8 -0.002 2 0.179 9 -0.032 9 -0.065 7 0.041
2 M:2 6 0.081 11 -0.052 7 0.077 6 0.101 8 0.019
1B:1 9 -0.040 10 -0.021 8 0.014 8 0.051 9 0.013
2(M+B):(1or2)OR(1or2) 11 -0.055 4 0.106 11 -0.058 11 -0.114 10 -0.008
2(M+B):(1or1)AND(1or1) 10 -0.041 6 0.065 10 -0.053 12 -0.162 11 -0.010
3 M:3 12 -0.127 13 -0.305 12 -0.147 10 -0.090 12 -0.125
2B:1or1 13 -0.228 12 -0.080 13 -0.204 13 -0.214 13 -0.164
2B:2and2 14 -0.331 14 -0.308 14 -0.287 14 -0.343 14 -0.307
Herds with a previous S. Dublin history
1(M+B):1or1 1 0.210 1 0.177 1 0.167 1 0.181 1 0.191
2(M+B):(1or1)OR(1or1) 3 0.141 2 0.163 4 0.083 4 0.107 2 0.149
1(M+B):1or2 2 0.161 3 0.114 2 0.128 3 0.121 3 0.132
1B:1 5 0.088 4 0.108 3 0.114 2 0.122 4 0.100
2(M+B):(1or2)OR(1or2) 7 0.060 5 0.083 8 0.037 7 0.030 5 0.070
3 M:1 4 0.089 6 0.042 7 0.045 6 0.052 6 0.061
1 M:1 6 0.060 8 0.020 5 0.068 5 0.053 7 0.034
1B:2 9 0.002 7 0.029 6 0.052 8 0.030 8 0.002
2 M:1 8 0.026 10 -0.035 10 -0.020 9 -0.003 9 -0.005
2(M+B):(1or1)AND(1or1) 11 -0.082 9 0.007 9 0.003 10 -0.041 10 -0.036
2 M:2 10 -0.065 11 -0.087 11 -0.038 11 -0.043 11 -0.067
2B:1or1 12 -0.151 12 -0.132 13 -0.195 13 -0.158 12 -0.142
3 M:3 13 -0.173 13 -0.192 12 -0.119 12 -0.123 13 -0.153
2B:2and2 14 -0.365 14 -0.298 14 -0.325 14 -0.328 14 -0.336
a
The code for the diagnostic strategies is as followed: the number and letter prior to the colon give the number of visits and the type of sample (i.e. blood of 10 young
animals (B), bulk milk (M) or both sample types (M+B)), respectively. The number of tests that must be positive to attribute a positive status to the herd depending on
the diagnostic strategy is after the colon (i.e. 1 bulk milk test, and/or 1 or 2 seropositive animals out of 10 tested per visit).

preferable in situations where the level of certainty in relation to a
negative result must be very high.
4. Discussion
4.1. Multi-criteria analyses results
The objectives of this study were to develop a MCDA tool to compare
diagnostic strategies aiming at defining a herd-level status for infectious
diseases in dairy herds, and to use that tool in the case of S. Dublin
infection in two epidemiological contexts. The various MCDA methods
have already been applied in animal health and veterinary public health,
but, so far, mainly to prioritize diseases or risks or to inform disease
prevention and control (Mourits et al., 2010; Corbellini et al., 2020;
Zhao et al., 2022; Amenu et al., 2023). Its use to inform decision on a
diagnostic strategy seems newer: we found one application to COVID-19
diagnosis in humans (Sayan et al., 2020) and another one for the peri­
odic screening of tuberculosis in bovine herds (Guetin-Poirier et al.,
2022).
The current study has shown how in a non-mandatory and complex
screening context for infectious diseases in dairies, a decision on diag­
nostic strategy can still be successfully informed by applying the MCDA
methodology. Here, we provided a transparent and documented 9-step
process, in the case of S. Dublin infection, starting with the working
team. The rigorous rainbow diagram tool allowed a balanced selection
for the MCDA team, which required all the S. Dublin project participants
to define the relevant stakeholders who must participate to the exercise
(from 18 to 7 stakeholders’ groups). In this manner, all the diagnostic
strategies included in the framework to define the herd status would
have the assent of the most impacted actors and actors who could in­
fluence the most their implementation in the field.
Moreover, the current framework has the additional advantage of
allowing a final brainstorming at the last step (consensual final deci­
sion). This highlights the fact that the objective of the MCDA exercise is
not a direct single recommendation, but a full consensual decision-
making process, helped and informed by the outputs of the analyses.
Interestingly, the main final decision that was adopted and consid­
ered the most relevant (1(M+B):1or1; i.e. testing the bulk tank milk and
the blood of 10 heifers >3-month-old, all samples taken during a single
farm visit and the herd being assigned a positive status if one sample is
ELISA-positive) happened to be already applied by the Québec’s au­
thorities (except for the milk-ELISA cut-off: set at PP% ≥ 35 in the
current study vs PP% ≥ 15 in the authorities policy) (MAPAQ, 2023).
Thus, this MCDA process provides consensual and rational bases to
support the diagnostic strategy already in place. It should be noted that
these multi-criteria results only apply to the defined options, criteria,
and disease contexts that were determined by the MCDA team. In case of
a new option (e.g. one including a different diagnostic test), the per­
formance matrix exercise should be carried out again. Moreover, an
additional diagnostic strategy (2(M+B):(1or1)OR(1or1)) was proposed
for the “with previous history of infection” context, notably in situations
where the decision-maker would like to minimize false negative results,
which demonstrates that final decision is not always unique and simple.
4.2. Method
Our MCDA team determined 10 criteria falling under four main is­
sues for comparing the diagnostic strategies. Guetin-Poirier et al. (2022)
worked in a similar way (i.e. the same MCDA steps and PROMETHEE II
tool were used) to inform decision on the periodic screening of tuber­
culosis in French bovine herds (Guetin-Poirier et al., 2022). Our two
studies had unique set of criteria and indicators. Nevertheless, they

9
M.M. Um et al. Preventive Veterinary Medicine 228 (2024) 106234

Fig. 3. GAIA plane showing the relative localization of the 14 diagnostic strategies and of the 10 criteria on the first two projection axes (UV plane). A/ For herds
with no previous Salmonella Dublin infection history (the representativity of the plane is 76%). B/ For herds with a previous Salmonella Dublin infection history (the
representativity of the plane is 75%). The diagnostic strategies are shown by red, pale blue and pink squares for the strategies based on blood (B), milk (M) or both
sample types (M+B), respectively. The code for the diagnostic strategies is as followed: the number and letter prior to the colon give the number of visits and the type
of sample, respectively. The number of tests that must be positive to attribute a positive status to the herd depending on the diagnostic strategy is after the colon (i.e.
1 bulk milk test, and/or 1 or 2 seropositive animals out of 10 tested per visit). The arrows ending with a diamond indicate the criteria, the color corresponding to their
issues of concern (gray: Diagnostic considerations; green: Socio-economic considerations; black: Logistics considerations; blue: Perception). The diamond indicates
the wanted optimal direction (i.e. minimizing or maximizing) for the criterion. The red stick highlights the decision axis for the consensus. The red ellipse shows the
location that the tip of the decision axis would cover if the weight of each criterion varied up to +/- 25% of its initial value. This allows assessing the sensitivity of the
decision axis to the weight of the criterion: the larger the ellipse, the more sensitive the decision axis is to the criteria weight.

share the similar main concerns about the decision. Regarding diag­
nostic considerations, the same authors (Guetin-Poirier et al., 2022)
used the diagnostic strategy’s sensitivity and specificity while we
included the probability of positive and of negative diagnostics (whose
computation includes the diagnostic sensitivity and specificity plus the
expected herd prevalence of disease). We chose those probabilities over
sensitivity and specificity because they are more informative regarding
the reliability of the test result when applied in a given epidemiological
context. This latter fact becomes obvious when comparing the varying
values for these criteria, within a same diagnostic strategy, but across
the two epidemiological contexts (i.e. in herds with vs without a previ­
ous S. Dublin history). As a comparison, the sensitivity and specificity of
a given diagnostic strategy, the criteria used by other authors would not
be modified by the epidemiological context. These latter parameters are
intrinsic characteristics of the test and they do not provide insights on
the test’s behavior when applied in a given population (Buczinski et al.,
2023).
In addition, we added credibility as another criterion regarding the
validity of the test strategy to weigh for the uncertainty around the es­
timates of the probabilities of positive and negative diagnostics. Finally,
we proposed and used a criterion on the sampling representativeness to
take into consideration the complex physiopathology of the disease. We
can find in the study by Guetin-Poirier et al. (2022) the same other
concerns about the socio-economics impacts, the logistics, and about
perception by farmers and veterinarians of the diagnostic strategy.
However, they are differently organized and defined. For example, two
criteria were used for the cost: one for the farmers and one for the au­
thorities (Guetin-Poirier et al., 2022). And their cost criteria encompass
not only the cost of the testing, as we included, but also the cost of the
control in case of positive animal, all of which being relevant because
tuberculosis is a notifiable disease with plan for culling positive animal.
The same authors considered the impact through their criterion “Per­
centage of animals culled for nothing” while we propose a more so­
phisticated yet validated approach (misclassification costs). They had
three criteria on the easiness of implementing the diagnostic strategy
(for the farmers, for the veterinarians, and for the authorities) which
partially overlap our criteria on resources and on farmers’ acceptability.
Sayan et al. (2020) in their multi-criteria evaluation of diagnostic tests
for COVID-19 in humans used 15 criteria, which were minimally defined
(Sayan et al., 2020). Nevertheless, one of them was about costs, four
about the diagnostics validity (sensitivity, specificity, false positivity,
false negativity), and one about the equipment, all being aligned to our
criteria. Although limited, these comparisons argue for the application
of MCDA methods in the choice of a diagnostic strategy to determine a
herd infection status when there is no obvious procedure. They also
indicate that such analyses should include criteria relating to the validity
of the diagnosis, but also criteria focusing on socio-economic impacts, on
logistical considerations, and on the perception by the different
stakeholders.
Given the relative repeatability of the issues and criteria across
studies, we hypothesize that an important part of the work conducted in
our study could be reused in future studies comparing diagnostic stra­
tegies for establishing the herd status for a given disease, when clinical
signs are not readily observable, and when a gold standard test is not
available and/or when it is impossible to sample all animals in the herd.
For such usage, one could use the issues, criteria, and indicators devel­
oped in our study. The development of those criteria and indicators (the
4th step of the MCDA process) was one of the most demanding part of
the MCDA exercise. The identification of the stakeholders (the first step),
as well as refining the decision problem (the 2nd step) and defining the

10
M.M. Um et al. Preventive Veterinary Medicine 228 (2024) 106234

Fig. 4. GAIA plane of the multicriteria preferences for the 14 diagnostic strategies by participating stakeholder. The red stick shows the decision axis: a) for herds
with no previous Salmonella Dublin infection history; b) for herds with a previous Salmonella Dublin infection history. The blue dots indicate the participating
stakeholders: the association of veterinary practitioners of Québec (AMVPQ); epidemiologists from the veterinary faculty (FMV); the ministry of agriculture. fisheries
and food of Québec (MAPAQ); the dairy producers of Québec combined with the private dairy herd improvement company in charge of individual and bulk milk
analyses and diagnostic in dairy cattle sector (PLQ). The options are shown by red, pale blue and pink squares for the strategies based on blood (B), milk (M) or both
sample types(M+B), respectively. The code for the diagnostic strategies is: the number and the letter prior to the colon give the number of visits and the type of
sample, respectively; the number of tests that has to be positive to attribute a positive status to the herd depending on the diagnostic strategy is after the colon (i.e. 1
bulk milk test and/or 1 or 2 seropositive animals out of 10 tested per visit).

options for the diagnostic strategies (the 3rd step) would, of course, have
to be disease and study-specific. In addition, the weights attributed to
each indicator (the 5th step) would also have to be determined to fit the
context of that new study. This latter matter is clearly illustrated by the
different weights that were attributed to the various indicators in our
study, when investigating the same disease (S. Dublin), but in different
contexts (herds with vs without previous S. Dublin history).
4.3. Limitations
The MCDA framework that was developed in the current study
resulted from the application of already well-described techniques
(Dodgson et al., 2009; Mareschal, 2013). With regards to the partici­
pants, a number of two key actors was selected by each stakeholders’
organization. Even though no fix number of stakeholders’ representa­
tives is recommended, we could have considered including more than
two representatives by organization. Our choice of options and contexts
can also be debated. Here, fourteen options were developed according to
Québec’s context, and to what is already in place in the province and in
northern European countries where the disease is endemic. Obviously,
this list is not exhaustive. Future studies should broaden the type of tests
and samples to be included in the diagnostic strategies (e.g. PCR, envi­
ronmental samples, etc.). Other epidemiological contexts could also
comprise the presence or absence of biosecurity measures in dairy herds.
Finally, another limitation to this study is the differences in the type
of variables that were used for criteria measurements. Most criteria (6/
10) were measured with strong indicators that are well known, repro­
ducible, and already published or well sourced (1_VD+, 2_VD-,
3_Credibility, 5_Costs, 6_Impacts, 7_Delay). However, the remaining
criteria were measured with terms that could be considered as “the
result of subjective interpretation” (4_Representativity, 8_Resources,
9_Acceptability, 10_Simplicity). Overall, it should be stressed that the
current proposed MCDA tool could be adapted for other situations, it
would have, however, to be modified according to the specific disease
and epidemiological context studied.
5. Conclusion
The present research proposed a multi-criteria decision analysis tool
that could be used to compare diagnostic strategies (i.e. combinations of:
biological samples, repetitions, and diagnostic test interpretations)
aiming at defining the herd status for infectious diseases in dairies. The
tool was then used in the case of S. Dublin infection in two contexts: (i)
herds with no history, and (ii) herds with previous history of infection.
The identified decision problem was “What is the optimal diagnostic
strategy for establishing the status of a dairy herd for S. Dublin infection
when there are no clinical signs of infection?”. Fourteen options of
diagnostic strategies were defined to help solving the problem. Ten
criteria grouped in diagnostic, socio-economic, logistic, and perception
considerations were selected to appraise the 14 diagnostic strategies.
The consensual final decision was 1(M+B):1or1 i.e. testing the bulk tank
milk and the blood of 10 heifers over 3-month-old, all samples taken
during a single farm visit and the herd being assigned a positive status if
one sample is ELISA-positive. These results are a first step in efforts to
help stakeholders with divergent priorities (e.g. dairy producers, vet­
erinarians, and authorities) overcoming diagnostic challenges with
regards to the screening of herds, notably for host-adapted diseases and
when the only available tests are imperfect.
Funding
This work was supported by funding from Les Producteurs de Lait du

11
M.M. Um et al.
Québec and a Collaborative Research and Development grant from the
Natural Sciences and Engineering Research Council of Canada (Grant
No. # CRDPJ: 518064-17) and the second author Natural Sciences and
Engineering Research Council of Canada Discovery grant (Grant No. #
RGPIN-2020-05237).
CRediT authorship contribution statement
Simon Dufour: Writing – review & editing, Writing – original draft,
Visualization, Validation, Supervision, Resources, Project administra­
tion, Methodology, Investigation, Funding acquisition, Formal analysis,
Data curation, Conceptualization. Maryse Michèle Um: Writing – re­
view & editing, Writing – original draft, Visualization, Project admin­
istration, Methodology, Investigation, Formal analysis, Data curation,
Conceptualization. Elouise Molgat: Writing – review & editing, Re­
sources, Methodology, Investigation. André Ravel: Writing – review &
editing, Writing – original draft, Visualization, Validation, Supervision,
Resources, Project administration, Methodology, Investigation, Funding
acquisition, Formal analysis, Data curation, Conceptualization. Marie-
Ève Paradis: Writing – review & editing, Resources, Methodology,
Investigation. Marie-Lou Gauthier: Writing – review & editing, Re­
sources, Methodology, Investigation. Myriam Falcon: Writing – review
& editing, Resources, Methodology, Investigation. Jean-Philippe Roy:
Writing – review & editing, Resources, Methodology, Investigation. Luc
Bergeron: Writing – review & editing, Resources, Methodology,
Investigation.
Declaration of Competing Interest
The authors declare the following financial interests/personal re­
lationships which may be considered as potential competing interests:
Simon Dufour reports financial support by the Natural Sciences and
Engineering Research Council of Canada and by Les Producteurs de Lait
du Québec.
Acknowledgments
Our special thanks go to all the members of the S. Dublin project,
namely @LIST OF NAMES REMOVED TO PRESERVE BLINDING OF
REVIEWERS@ for their different involvement in this study.
Appendix A. Supporting information
Supplementary data associated with this article can be found in the
online version at doi:10.1016/j.prevetmed.2024.106234.
References
Aenishaenslin, C., Hongoh, V., Cisse, H.D., Hoen, A.G., Samoura, K., Michel, P.,
Waaub, J.P., Belanger, D., 2013. Multi-criteria decision analysis as an innovative
approach to managing zoonoses: results from a study on Lyme disease in Canada.
BMC Public Health 13, 897.
Agren, E.C.C., Lewerin, S.S., Frossling, J., 2018. Evaluation of herd-level sampling
strategies for control of Salmonella in Swedish cattle. J. Dairy Sci. 101,
10177–10190.
Amenu, K., McIntyre, K.M., Moje, N., Knight-Jones, T., Rushton, J., Grace, D., 2023.
Approaches for disease prioritization and decision-making in animal health, 2000-
2021: a structured scoping review. Front. Vet. Sci. 10, 1231711.
Behzadian, M.K., Albadvi, A.R.B., Aghdasi, M., 2010. PROMETHEE: a comprehensive
literature review on methodologies and applications. Eur. J. Oper. Res., 200, pp.
198-215.
Brans, J.-P., De Smet, Y., 2016. PROMETHEE methods. In: Salvatore Greco, M.E., José
Rui Figueira (Ed.), Multiple criteria Decision Analysis: State of the Art Surveys.
Springer New York, NY, New York, NY, pp. 187–219.
Brans, J.P., Vincke, P., Mareschal, B., 1986. How to select and how to rank projects - the
Promethee method. Eur. J. Oper. Res. 24, 228–238.
Buczinski, S., Dufour, S., Arango-Sabogal, J.C., 2023. Interpretation and analysis of
individual diagnostic tests and performance. Vet. Clin. N. Am. Food Anim. Pract. 39,
1–19.
Chevalier, J.M., Buckles, D.J., 2008. A Guide to Collaborative Inquiry and Social
Engagement. Ebook available here: https://idl-bnc-idrc.dspacedirect.org/server/
12
Preventive Veterinary Medicine 228 (2024) 106234
api/core/bitstreams/9c4fe1f8-24b9-47a0-852a-a5bed717a927/content. Seen on
November, 2023. SAGE Publications India Pvt Ltd New Delhi, India.
Corbellini, L.G., Fernandez, F., Vitale, E., Moreira Olmos, C., Charbonnier, P., Iriarte
Barbosa, M.V., Riet-Correa, F., 2020. Shifting to foot-and-mouth disease-free status
without vaccination: application of the PROMETHEE method to assist in the
development of a foot-and-mouth national program in Uruguay. Prev. Vet. Med. 181,
105082.
Cox, R., Sanchez, J., Revie, C.W., 2013. Multi-criteria decision analysis tools for
prioritising emerging or re-emerging infectious diseases associated with climate
change in Canada. PLoS One 8, e68338.
Dodgson, J.S.S., Pearman, A.M., Phillips, L.D., 2009. Multi-Criteria Analysis: A Manual.
Available here: https://eprints.lse.ac.uk/12761/1/Multi-criteria_Analysis.pdf.
London School of Economics and Political Science, Department of Economic
History..
Drewe, J.A., Snary, E.L., Crotta, M., Alarcon, P., Guitian, J., 2023. Surveillance and risk
assessment for early detection of emerging infectious diseases in livestock. Rev. Sci.
Tech. 42, 120–127.
Greco, S., Ehrgott, M., Figueira, J.R., 2016. Multiple Criteria Decision Analysis: State of
the Art Survey. Springer, New York, NY. https://doi.org/10.1007/978-1-4939-3094-
4.
Guetin-Poirier, V., Dufour, B., Riviere, J., 2022. A framework for multicriteria decision-
aid analyses in animal health surveillance applied to periodic screening for French
bovine tuberculosis. Transbound. Emerg. Dis. 69, 1281–1293.
Holschbach, C.L., Peek, S.F., 2018. Salmonella in dairy cattle. Vet. Clin. N. Am. Food
Anim. Pract. 34, 133–154.
Hongoh, V., Gosselin, P., Michel, P., Ravel, A., Waaub, J.P., Campagna, C., Samoura, K.,
2017. Criteria for the prioritization of public health interventions for climate-
sensitive vector-borne diseases in Quebec. PLoS One 12, e0190049.
Jordan, D., Nielsen, L.R., Warnick, L.D., 2008. Modelling a national programme for the
control of foodborne pathogens in livestock: the case of Salmonella Dublin in the
Danish cattle industry. Epidemiol. Infect. 136, 1521–1536.
Keeney, R.L., Raiffa, H., 1976. Decisions with Multiple Objectives: Preferences and Value
Tradeoffs. Cambridge University Press, New York, NY, US.
MAPAQ, 2023. Ministère de l’Agriculture, des Pêcheries et de l’Alimentation (Ministry of
agriculture, fisheries, and foof of Quebec). Surveillance, prévention et contrôle des
infections à Salmonella Dublin dans les élevages laitiers et vache-veau. Janvier 2023.
Available on: https://cdn-contenu.quebec.ca/cdn-contenu/adm/min/agriculture-
pecheries-alimentation/sante-animale/surveillance-controle/raizo/reseau-bovin/
FS_salmonella_vache_veau_MAPAQ.pdf. Seen on November 2023.
Mareschal, B., 2013. Visual PROMETHEE 1.4 Manual. Available here: https://en.
promethee-gaia.net/assets/vpmanual.pdf. Seen on December, 2022. 2012-2013 by
VPSolutions.
Mourits, M.C., van Asseldonk, M.A., Huirne, R.B., 2010. Multi criteria decision making to
evaluate control strategies of contagious animal diseases. Prev. Vet. Med. 96,
201–210.
Nemeth, B., Molnar, A., Bozoki, S., Wijaya, K., Inotai, A., Campbell, J.D., Kalo, Z., 2019.
Comparison of weighting methods used in multicriteria decision analysis
frameworks in healthcare with focus on low- and middle-income countries. J. Comp.
Eff. Res. 8, 195–204.
Nielsen, L.R., 2013. Review of pathogenesis and diagnostic methods of immediate
relevance for epidemiology and control of Salmonella Dublin in cattle. Vet.
Microbiol. 162, 1–9.
Nielsen, L.R., Houe, H., Nielsen, S.S., 2021. Narrative review comparing principles and
instruments used in three active surveillance and control programmes for non-EU-
regulated diseases in the danish cattle population. Front. Vet. Sci. 8, 685857.
Nobrega, D.B., French, J.E., Kelton, D.F., 2023. A scoping review of the testing of bulk
milk to detect infectious diseases of dairy cattle: diseases caused by bacteria. J. Dairy
Sci. 106, 1986–2006.
RAIZO, 2022. Réseau d’alerte et d’information zoosanitaire (Quebec’s animal health
alert and information network). Bilan du réseau bovin 2022. Available here: https://
cdn-contenu.quebec.ca/cdn-contenu/adm/min/agriculture-pecheries-alimentation/
sante-animale/surveillance-controle/raizo/reseau-bovin/RA_bilan_bovin_2022_
MAPAQ.pdf. Seen on November 2023. In: MAPAQ (Ed.).
Rasmussen, P., Barkema, H.W., Mason, S., Beaulieu, E., Hall, D.C., 2021. Economic losses
due to Johne’s disease (paratuberculosis) in dairy cattle. J. Dairy Sci. 104,
3123–3143.
Sayan, M., Sarigul Yildirim, F., Sanlidag, T., Uzun, B., Uzun Ozsahin, D., Ozsahin, I.,
2020. Capacity evaluation of diagnostic tests for COVID-19 using multicriteria
decision-making techniques. Comput. Math. Methods Med. 2020, 1560250.
van Til, J., Groothuis-Oudshoorn, C., Lieferink, M., Dolan, J., Goetghebeur, M., 2014.
Does technique matter; a pilot study exploring weighting techniques for a multi-
criteria decision support framework. Cost Eff. Resour. Alloc. 12, 22.
Um, M.M., Castonguay, M.H., Arsenault, J., Bergeron, L., Cote, G., Fecteau, G.,
Francoz, D., Giguere, J., Amine, K.M., Morin, I., Dufour, S., 2022. Estimation of the
accuracy of an ELISA test applied to bulk tank milk for predicting herd-level status
for Salmonella Dublin in dairy herds using Bayesian Latent Class Models. Prev. Vet.
Med. 206, 105699.
Um, M.M., Castonguay, M.H., Arsenault, J., Bergeron, L., Fecteau, G., Francoz, D.,
Dufour, S., 2023. Accuracy of testing strategies using antibody-ELISA tests on
repeated bulk tank milk samples and/or sera of individual animals for predicting
herd status for Salmonella Dublin in dairy cattle. Prev. Vet. Med. 220, 106048.
Um, M.M., Castonguay, M.-H., Mahamad Amine, K., Giguère, J., Morin, I., Dufour, S.,
2020. Repeatability of a commercially available ELISA test for determining the herd-
M.M. Um et al. Preventive Veterinary Medicine 228 (2024) 106234

level Salmonella enterica subsp. enterica serovar Dublin status in dairy herds using Zhao, J., Smith, T., Lavigne, M., Aenishaenslin, C., Cox, R., Fazil, A., Johnson, A.,
bulk milk. Front. Vet. Sci. 7. Sanchez, J., Hermant, B., 2022. A rapid literature review of multi-criteria decision
Wiecek, M.M., Ehrgott, M., Fadel, G., Figueira, J.R., 2008. Multiple criteria decision support methods in the context of one health for all-hazards threat prioritization.
making for engineering. Omega 36, 337+. Front. Public Health 10, 861594.

13