International

Review of
Administrative
Article Sciences

International Review of Administrative

Sciences
Media coverage of 2021, Vol. 87(2) 347–363
! The Author(s) 2019
reports published by the Article reuse guidelines:
sagepub.com/journals-permissions
Québec Ombudsman: DOI: 10.1177/0020852319870241
journals.sagepub.com/home/ras

an automated
content analysis

Dominic Duval
Université du Québec à Montréal, Canada

Steve Jacob
Université Laval, Canada

Eric Montigny
Université Laval, Canada

Mathieu Ouimet
Université Laval, Canada

Abstract
This article examines media coverage of reports published by the Québec Ombudsman,
a body that upholds the rights of citizens and that goes by the name of ‘Public
Protector’. A large part of the Québec Ombudsman’s mandate is to conduct inves-
tigations and issue recommendations following infringements by Québec’s administra-
tive apparatus that affect one or several citizens. These infringements are reported to
the ombudsman by citizens, which means that it must be visible to the public.
Such visibility relies, to a great extent, on the media, hence the importance of analysing
the Québec Ombudsman’s media coverage, a subject that has received little attention in
the academic literature. Our article reveals that media coverage of the ombudsman’s
reports is inconsistent. We also observe that, on average, newspaper articles adopt a
more negative tone than the reports themselves. However, contrary to our

Corresponding author:
Dominic Duval, Département de communication sociale et publique, Université du Québec à Montréal, 405
rue Sainte-Catherine Est, Montréal, Québec, Canada H2L 2C4.
Email: duval.dominic.2@uqam.ca
348 International Review of Administrative Sciences 87(2)

expectations, reports with a more negative tone are not necessarily given more media
coverage. The best predictor of the presence or absence of media coverage and tone
congruence between reports and articles appears to be the presence of a press release
issued by the ombudsman.

Points for practitioners

This article examines media coverage of reports published by the Québec Ombudsman.
Based on an automated content analysis, it appears that the media coverage of the
reports is not explained by the tone used in the documents published by the Québec
Ombudsman. Reports that are more negative are not necessarily given greater cover-
age by journalists than positive reports. Direct communication efforts with the media
(e.g. a press conference and the publication of a press release) are more likely to lead to
media coverage.

Keywords
administrative reports, media coverage, ombudsman, Public Protector

Introduction
Administrative reports are tools to inform and communicate produced by public
organisations whose common denominator is that they ‘draw on the current state
of a sector, public policy or issue . . . to propose avenues for reflection and/or
action’ (Dupuy and Pollard, 2012: 6). Since the reports are a ‘showcase’, the
requirements for producing these documents are often regulated by legislative
obligations or requirements specific to each public organisation (Maingueneau,
2002: 119). When it is made public, the administrative report remains ‘silent on
the hierarchies and successive rewritings which possibly contributed to it’ (Gayon,
2016: 92). As a general rule, bureaucratic writing is formalised by guidelines on
writing instructions and restrictive internal content validation conventions that are
based on a posture of impartiality, neutrality and objectivity in order to guarantee
its legitimacy (Fournel, 2007; Gayon, 2016; Laurent and Sanson, 1996). The ‘tone
[of the reports is] often very descriptive, even flat . . . polarisation is avoided,
variations are tempered, dramatic adjectives are removed, categories and bureau-
cratic notions disseminated by the hierarchy are employed’ (Penissat, 2012: 56).
This material, which is essential in public administration research, is rarely the
subject of the exclusive analysis that we envisage this research to be: media analysis
of the reports produced by the Québec Ombudsman (or the Protector). This insti-
tution, created in 1968, follows the classic Scandinavian model of Ayeni’s (1985)
typology. In practical terms, the ombudsman is a person appointed by the
National Assembly1 whose mission is to ensure that individual rights are respected
by addressing the ministries and agencies of the government of Québec, as well as
Duval et al. 349

various authorities within the health and social services network, to request cor-
rective action in situations harmful to a citizen or a group of citizens. It is an
impartial and independent institution that operates a system for handling com-
plaints in order to defend citizens’ rights against the government.
At its inception, the Swedish ombudsman carried out a mission of legal medi-
ation that gradually integrated the defence of rights (Simonian-Gineste, 2015), or
more specific mandates such as access to information. In Québec, the ombudsman
has also evolved into a hybrid model and has recently been granted new powers
covering the integrity and disclosure of wrongdoing by public officials. From this
perspective, the ombudsman serves as a bulwark against maladministration and
provides a counterweight to bureaucratic power, which is sometimes portrayed as
cold and disembodied (Gregory and Giddings, 2000). The ombudsman takes on
the role of monitoring the democratic state, often referred to using the graphic
term ‘watchdog’. One feature that distinguishes the Québec Ombudsman from
other ‘watchdogs’, such as the auditor general, is that the ombudsman responds
directly to citizens’ requests (Affodegon et al., 2017). It should be noted that its
constituent law also allows it to intervene on its own initiative. As such, the
Ombudsman conducts investigations, proposes corrective action and writes
reports. In its most modern version, the ombudsman also contributes to establish-
ing administrative reforms. To fulfil this mission, the ombudsman draws on the
findings of its investigations and formulates proposals for reform that it must then
bring to fruition through the public policymaking process (Baudot and Revillard,
2011: 345–346).
When the ombudsman wishes to correct a specific administrative injustice or
repair a systemic, common problem, the ombudsman has several resources at its
disposal. The most obvious is the power of its influence. However, when the latter
is ineffective, the ombudsman may then publicise the situation to force the admin-
istrative authorities to reconsider their decision or review their procedures.
The ‘mobilisation of shame’ is presented as a key element in the ombudsman’s
arsenal (Giddings, 2000; Giddings et al., 2011). In Québec, we note that the
ombudsman can only act within a certain defined framework since it has no
enforcement power, only the power of persuasion. The Québec Ombudsman’s
investigations are conducted in a confidential manner and its findings are generally
not made public. However, in the case of systemic investigations and health and
social services actions carried out in the wake of a report or on its own initiative,
the institution publishes a report containing a number of recommendations.
Like other stakeholders seeking to place an issue on the government’s agenda,
the ombudsman can reach out to the media to make its proposals more visible
(Cantelli et al., 2006; Wing-Hung Lo et al., 2011). Due to its specific position
within the institutional structure, its voice carries more weight than that of
others within the political-administrative system and is sometimes seen as the
voice of the ‘voiceless’ (Lezertua, 2018). Many ombudsman offices issue press
releases to inform the public about specific issues (Bernt and Owen, 2000; Reif,
2004). The media can thus play its part in putting a more complex issue on the
350 International Review of Administrative Sciences 87(2)

political agenda. The effect of this is to exert additional pressure on organisations

targeted with a recommendation to more quickly resolve a problematic situation.
A few examples that have received media coverage illustrate the themes or issues
studied by the Québec Ombudsman. In terms of programme or public policy
management, these could be tax assessments not supported by evidence from the
Revenue Agency or unexplained delays in delivering a service, compensation or
support provided by a government organisation. In terms of the quality of admin-
istrative services, the ombudsman may also identify gaps in services offered by
housing resources for people with disabilities or for young people at risk.
Although all of the ombudsman’s activities are not limited to investigating
complaints from citizens, it can nevertheless be argued that the exercise of the
ombudsman’s mandate depends, among other things, on its visibility to the
public (Fortier, 2009). Lee Mordicai (2005: 240) argues that an administrative
report ‘focuses on efforts to contribute to an informed public. In that sense,
reporting contributes to the functioning of democracy.’ However, a number of
studies suggest that when it comes to staying informed, citizens do not tend to
consult official documents issued by political parties or the government, such as
Bills or budgets (Andersen et al., 2005; Walgrave and De Swert, 2007). Rather,
they refer to information reported in the media. This research suggests that admin-
istrative reports may not be created with the general public in mind.
In fact, the reports of the Québec Ombudsman are sometimes very technical,
contain many legal or methodological elements, and are not easy for the general
public to read. Not surprisingly, the exposure a report receives, at least in terms of
its visibility to the public, therefore depends mainly on traditional media and the
social networks on which the ombudsman relies to communicate the highlights of
its reports. This research focuses on the media coverage of the reports produced by
the ombudsman. There are very few studies on the media coverage of the Public
Protector in Québec or ombudsmen in general (Monogioudis, 2015), and we
believe that we can rely on other research in the field of political communication
to guide our efforts. In the next section, we will discuss this work. Based on the
findings we have just presented, we expect the tone of the ombudsman’s reports to
be neutral for two reasons: first, because it is one of the fundamental characteristics
of administrative reports; and, second, because the Québec Ombudsman’s credi-
bility with the public and political-administrative decision-makers is essential to
accomplishing the ombudsman’s mission, which depends on a position of neutral-
ity (Baudot and Revillard, 2011; Brothers, 2014; Gregory and Giddings, 2000).

Issue
The media have earned a reputation for focusing on negative news. We only have
to think of expressions like ‘if it bleeds, it leads’ by Pooley or ‘the real news is bad
news’ by McLuhan. This phenomenon has been widely discussed and studied.
Take, for instance, Patterson’s (1993) book, Out of Order, which describes the
media as increasingly sensationalistic and negative towards politics, or that by
Duval et al. 351

Farnsworth and Lichter (2007), which shows that the tone of televised political
news in the US became increasingly negative between 1988 and 2004. This attitude
adopted by the media can be negatively perceived as a source of cynicism and
mistrust of the population towards political and government leaders. Conversely,
by focusing on what does not work, the media helps to improve how public
organisations function as they change their practices when held to account (learn-
ing by mistakes).
More recently, Soroka (2014) made two comparisons showing this phenome-
non: the first compared the quantity of crimes committed with the quantity of
articles on these crimes; the second compared the state of the economy with the
tone of economic news. In both cases, the media coverage observed is significantly
more negative than the facts suggest. The media gave considerable importance to
negative events, suggesting that crime and the economy were significantly worse
than the statistical indicators would suggest. This negative bias is also found in
subjects closer to ours. For example, Barlett et al. (2002) have shown that ‘omi-
nous’ medical discoveries are much more likely to be discussed in newspapers than
discoveries that can be described as ‘good news’. They measured media coverage of
1193 scientific articles published in The Lancet and the British Medical Journal and
found that ‘bad news’ articles are almost twice as likely to be covered by print
media in England.
Although our main body of texts does not consist of scientific articles or news
focusing on electoral politics, but rather on reports of investigations conducted by
the Québec Ombudsman, we have no reason to believe that our results will differ
from those of the studies briefly presented earlier. The assumptions are as follows:

1. The tone of media coverage will be more negative than the tone of the reports.
2. Reports that have a more negative tone will receive more media coverage.

Empirically, these expectations are mapped in the graphs in Figure 1, which is

borrowed (and adapted) from Soroka’s (2014: 84) Negativity in Democratic
Politics. The tone distribution of the reports should be very centred, like that of
sub-graph 1. As for the tone distribution of newspaper articles, they should shift to
the left, which would mean that the tone of media coverage is more negative, as
observed in sub-graph 3. Sub-graph 2 illustrates the theoretical probability that an
item will be covered based on its tone. If the graph is very negative (left side), then
the probability will be very high and this probability will gradually decrease as the
tone becomes more positive (shift of the curve to the right).

Method
Document collection
Reports published between 2009 and 2017 were downloaded from the ombuds-
man’s website by a research assistant. He then changed them from the .pdf format
352 International Review of Administrative Sciences 87(2)

Tonalité des rapports

(Real World Phenomena)

Densité

Négative Neutre Positive

Tonalité
Sélection
(Gatekeeping Function)
Probabilité d'être couvert

Négative Positive
Tonalité

Tonalité de la couverture des rapports

(Media coverage)
Densité

Négative Neutre Positive

Tonalité

Tonalité des rapports Report Tone

Densité Density
Négative Negative
Neutre Neutral
Tonalité Tone
Positive Positive
Sélection Selection
Probabilité d’être couvert Probability of being covered
Tonalité de la couverture des rapports Report coverage tone

Figure 1. Mapping of theoretical expectations.

Duval et al. 353

used to a .txt format in order to make them machine-readable. A total of eight

annual reports, 22 special reports and 42 intervention reports were collected. The
annual report is an important document that takes stock of the situation in the
Québec Ombudsman’s fields of expertise, while making recommendations to
improve services to citizens. It is tabled in the National Assembly every year.
Special reports are the result of investigations that the Québec Ombudsman has
undertaken on its own initiative, specifically, on government departments and
agencies ‘when it sees widespread harmful situations’ (Québec Ombudsman, web-
site).2 These reports are far-reaching and preventive in nature. It is also possible to
publish intervention reports that focus only on the health and social services
sector.3 Reports may be created following notification or on the ombudsman’s
own initiative.
Daily press articles were collected from Eureka.cc before being modified. In order
to target all articles dealing with the ombudsman, the following search terms were
used: (‘Québec Ombudsman’ & ‘report*’) jj (‘Québec Ombudsman’ & ‘investiga-
tion*’) jj (‘Québec Ombudsman’ & ‘recommendation*’) jj (‘Ombudsperson’ &
‘report*’) jj (‘Ombudsperson’ & ‘investigation*’) jj (‘Ombudsperson’ & ‘recommen-
dation*’). The research spans the period from 1 June 2009 to 30 June 2017. The
journalistic sources are as follows: Le Devoir, Le Quotidien, Le Nouvelliste, Le Soleil,
La Presse and la Presse þ, La Voix de (Granby), La Tribune (Sherbrooke),
Le Journal de Montréal, Le Journal de Que´bec, Métro (Montréal) and 24 heures
Montréal. Our body of articles contains 748 articles.

Analyses
In order to assign reports and articles a tone expressed as a numerical value, we
used ‘Lexicoder Sentiment Dictionnaire Français’ (LSDFr) (Duval and Pétry,
2016), which is a dictionary used for automated analysis. It was developed from
Young and Soroka’s (2012) English lexicon, the Lexicoder Sentiment Dictionary.
The LSDFr consists of two long lists of words, one with positive and the other with
negative connotations. The dictionary contains 2867 negative words and 1284
positive words. In order to gauge the tone of a document, we subtract the
number of negative words from the number of positive words that it contains.
We then divide this difference by the total number of words in the text to eliminate
the effect attributable to the length of the text. The validity of this analytical
approach has been established. Duval and Pétry did so by comparing it with a
body of texts whose tone was manually coded by three coders. Their results clearly
illustrate that the scores resulting from the automated dictionary-based analysis
are highly comparable to those of human coders. Indeed, articles coded as negative
by coders obtained an average tone of –0.011 with the LSDFr, neutral articles
obtained a tone of 0.006 and positive articles obtained a score of 0.042 (Duval and
Pétry, 2016).
Basing this analysis approach on a pre-established lexicon has certain advan-
tages and disadvantages. The main advantage of automated analysis is the time
354 International Review of Administrative Sciences 87(2)

saved. This makes it possible to analyse subjects that require a large body of texts
to be studied. A second advantage is that it gives a completely accurate reading: the
results of several independent codings carried out using the same method on the
same body of texts by different operators are perfectly consistent, which is not
the case with human coders.
The main disadvantage of the automated classification method using a dictionary
is that it does not take into account context, figures of speech, negations, double
negatives and so on. It is a summary analysis that is less able to capture complexity
than a trained human coder would. The second disadvantage is that dictionaries are
developed in order to be used with a specific type of body of texts. For example, a
dictionary designed to analyse public policies must be adapted before it can be
applied to press releases. This is the case here: since the LSDFr was mainly devel-
oped to analyse newspaper articles, we had to remove the words in the dictionary
that lost their negative or positive connotation in the ombudsman’s reports in order
to make it suitable for the analysis of the latter. For example, ‘recommended/rec-
ommendation’ is commonly used in reports without a positive connotation, and the
word ‘patient’ does not refer to the quality, but rather refers to users of health
services. A total of 32 words were removed from the dictionary.
Our various documents were imported into the R statistical software using the
Quanteda content analysis suite developed by political scientist Kenneth Benoit
et al., (2018). The body of texts was then cleaned: the ‘stop terms’, that is, mean-
ingless words such as pronouns, articles, common verbs and so on, were removed,
as were numbers and word endings (lemmatisation) in order to remove conjuga-
tions and agreements. The goal is that, for example, both ‘excellent’ and ‘excel-
lente’ in French be counted as the same word. After cleaning, we counted the
frequency of words found in the dictionary and then calculated the tone of each
article and report as described earlier ((Nb positive – Nb negative)/Nb total)). As
an example, consider the following sentence: ‘The Québec Ombudsman has
received reports of private residences housing people with mental health problems:
lack of supervision, problematic behaviour of owners, poor management of med-
ication, unsanitary premises.’ This sentence from the 2014 annual report would
score –0.103 since it contains no positive codewords, three negative codewords
(underlined in the excerpt) and a total of 29 words after cleaning the stop terms
((0–3)/29 ¼ –0.103). Another example, which, this time, comes from media cover-
age of the same report, is as follows: ‘We would like to express our outrage at the
incompetence of public servants, describe the actions denounced by the Québec
Ombudsman as unforgivable errors.’ This sentence would score –0.308 since it
contains no positive words, four negative words and a total of 13 words after
cleaning the stop terms ((0–3)/13 ¼ –0.308).

Results
A total of 748 media articles were collected. Of these articles, 310 cite a specific
report while 438 mention the Québec Ombudsman but do not mention a specific
Duval et al. 355

report. Considering that we collected articles covering 72 reports, this equates to

an average of 4.3 articles per report. However, media coverage of the ombuds-
man’s reports is inconsistent. Indeed, the different types of reports collected did
not receive the same attention from the media. In total, the eight annual reports
were reported on 142 times, with an average of 17.7 articles per report. The
22 special reports were reported on 164 times, with an average of 7.4 articles per
report. Finally, the 42 intervention reports were cited 40 times, with an average of
0.9 articles per report. Many of these differences can be explained by the fact that
not all reports were covered in the media. Indeed, while 64% of intervention
reports (27 of 42) and 18% of the special reports (4 of 22) did not receive media
coverage, all annual reports were the subject of at least one newspaper article.
The average tone of the reports is 0.0072 (standard deviation: 0.009). Again, we
see variations by report type. The average tone of the annual reports is 0.0016
(0.003), the average tone of the special reports is 0.0043 (0.012) and the average
tone of the intervention reports is 0.0099 (0.009). The different standard deviations
presented in brackets also illustrate the variations in the tone of the articles. We
must not only take into consideration the size of the figures, which seems very
small. With a tool like a Lexicoder Sentiment Dictionary (LSD), we are more
interested in variations and comparisons than in absolute values. The tone of
intervention reports is significantly more positive than those of other types of
report. Indeed, the Welch t-test result is t ¼ –5.5 (p < .05) when comparing averages
between special and intervention reports, and t ¼ –1.9 (p < 0.1) when comparing
averages between annual and intervention reports (though there are only eight
annual reports).
It is important to remember that our first hypothesis states that the tone of
media coverage should be more negative than the tone of the reports themselves.
Figure 2 presents this comparison. The two lines correspond to the tone distribu-
tions of the reports (solid line) and newspaper articles (dotted line) covering these
same reports. This visualisation presents the equivalent of sub-graphs 1 (top) and 3
(bottom) of our theoretical expectations shown in Figure 1.
We observe that the tone of the reports is significantly more positive than that of
the newspaper articles. Indeed, the average tone of the 41 reports covered in the
newspapers is 0.0065 (standard deviation: 0.011), while the average tone of the
newspaper articles covering these same reports is –0.0079 (0.017). The difference
between the two averages is statistically significant (t ¼ 4.34 (p < 0.001)). We also
observe that the tone of reports is much more stable than that of newspaper
articles. The tone distribution of the reports is leptokurtic (‘more peaked’;
kurtosis ¼ 6.01) while the tone distribution of the media articles is flatter
(kurtosis ¼ 3.64). In other words, the results are more concentrated around the
same values in the case of Québec Ombudsman reports than in the case of news-
paper articles.
Our second hypothesis is that more negative reports should receive more media
attention than more positive reports. Figure 3 compares the tone of reports men-
tioned in the media with the tone of reports that received no media coverage.
356 International Review of Administrative Sciences 87(2)

50
Tonalité des rapports
40
30 Tonalité de la couverture médiatique
Densité

20
10
0

−0.04 −0.02 0.00 0.02 0.04

Tonalité

Figure 2. Tone of reports and their media coverage.

In theory, the tone of the reports covered should be more negative than the tone
of the reports not covered. This is what the figure seems to indicate, the dotted line
being slightly more to the right than the solid line. Despite this slight difference, the
two distributions are practically the same. Indeed, the difference observed visually
is not statistically significant (t ¼ –0.67 (p ¼ 0.50)). Nor is this lack of difference
caused by the presence of intervention reports since even when they are excluded,
the results remain the same.
Figure 3 presents a somewhat simplistic distinction between types of reports.
Media coverage is operationalised as a dichotomous variable, that is, the presence
or absence of an article citing a report. If we take into account the volume of
articles published rather than this binary variable, we can correlate the tone of the
reports covered with the number of articles later resulting from them. This addi-
tional analysis is necessary because it would be conceivable that once the ‘media-
entry barrier’ has been crossed, a more negative report would generate more ink
than a more positive report. That is not what is observed. The correlation between
the tone of the reports covered and the number of articles discussing these reports
is r ¼ –0.14, which is contrary to our theoretical expectations.
If tone does not explain the media coverage of a report, then how can it be
explained? Our data indicate that the communication efforts made by the Québec
Duval et al. 357

Tonalité des rapports couverts

50
40
30 Tonalité des rapports non couverts
Densité
20
10
0

−0.04 −0.02 0.00 0.02 0.04

Tonalité

Figure 3. Tone of covered and uncovered reports.

Ombudsman may have more impact than the tone of the reports. Obviously, it is
difficult to assess the energy deployed by the Québec Ombudsman with the media
for each of its interventions. This usually takes the form of telephone calls, emails
to journalists or even press conferences. However, we do have the press releases
issued by the Québec Ombudsman and can use them as indicators of the commu-
nication efforts made for some reports.
Therefore, of the 72 reports analysed, only 26 were accompanied by press
releases. More specifically, eight are annual reports accompanied by press confer-
ences and documents given to journalists, while 18 other reports were announced
only by a press release. Of these 18 press releases, 17 addressed special reports and
only one addressed an intervention report. Of the 22 special reports, 17 (77%) were
publicised through a press release.
The communication strategy used by the ombudsman therefore differs according
to the type of report. A report addressing a systemic problem will therefore be more
likely to be given communicational support than a report addressing an individual
problem. In the latter case, the problem is generally solved through a targeted
intervention with the organisation concerned. On the other hand, a recommendation
of systemic scope is based on a cluster of several complaints or collective complaints.
At times, it may also involve several organisations. Since these issues are generally
358 International Review of Administrative Sciences 87(2)

more complex to resolve, the use of a press relations strategy can help to make things
happen more quickly by putting the media on the case. It should also be noted that
distributing a press release also makes it possible to disseminate the more technical
elements of a report, which facilitates journalistic work.
The use of a press release does increase the likelihood of media coverage. Out of
18 press releases, we identified articles discussing 14 (78%) of them. Only 58% of
the reports that were not the subject of a press release (excluding annual reports
that are the subject of several other communication exercises) were covered in the
media. This is a difference of 20 percentage points.
Press releases also appear to have significant impact on the extent of media cov-
erage. On average, the 18 reports that were the subject of a press release were cov-
ered by 11.7 journalistic articles (standard deviation ¼ 16.18), while the average for
all reports is only 4.3 (standard deviation ¼ 9.57). As for tone, when there is a press
release, the tone of the reports correlates more strongly with the tone of the articles
than when there are no press releases. Indeed, the tone of the 27 reports covered that
were not the subject of a press release correlates weakly with the tone of the articles
(r ¼ –0.56). On the other hand, when we correlate the tone of our 14 covered articles
that were the subject of a press release, it is much stronger (r ¼ 0.76). The presence of
a press release therefore seems to ‘set the tone’ for the following articles.
In short, the media coverage of the reports is, indeed, much more negative than
the reports themselves, as proposed in our first hypothesis. However, our second
hypothesis, which proposed that more negative reports receive more media atten-
tion, is not supported by our data and analyses. The ombudsman’s communication
efforts seem to have a greater impact on media coverage.

Discussion
In the introduction, we emphasised that the tone of the administrative reports was
very descriptive, even flat. On the basis of our analyses, this statement must be
qualified since the Québec Ombudsman’s reports are in line with the findings of
Truan and Renard (2017: 90), who present the reports as ‘strictly political objects
. . . caught between a double contradictory injunction: informing public opinion,
subject to confusion, and influencing public action’. To achieve this, the media is
an important intermediary actor.
The fact that more negative reports do not generate more coverage is quite sur-
prising considering that negative bias is found in almost all media content studies
(see Soroka, 2014). This may suggest that journalists, often in a hurry and with
several articles to write per day, probably do not have the time, or the desire, to
carefully read the entire content of voluminous and often technical reports such as
those from the Québec Ombudsman. To produce news quickly, they may rely more
on the outputs of the ombudsman’s communications branch, such as press releases.
In this case, from a media perspective, the structuring of the report by the commu-
nications branch would become just as important as the content of the report.
Duval et al. 359

In addition, an evaluative study examining the relationships of Asian ombudsmen

with different stakeholders such as the media shows that there are significant varia-
tions between ombudsmen, some of whom do not maintain any relationship with the
media (Wing-Hung Lo et al., 2011). This invites us to be cautious in generalising our
results. In addition, this study identifies a plurality of ways in which ombudsmen enter
into dialogue with the media, ranging from press conferences to informal communi-
cations. It should also be noted that an ombudsman has relationships with several
other stakeholders, including interest groups, and that its relationships with these
stakeholders are also likely to influence the tone of its reports. The same is true of
the media, which is sometimes targeted by interest groups or lobbyists who seek to
influence them (Binderkrantz, 2008). The tone of media articles covering the reports
of organisations such as the Québec Ombudsman can therefore be explained by a
multitude of factors beyond the tone of its reports or its relations with the media. The
combined effect of these different factors not measured in our study could explain
why reports that are more negative are not covered more extensively in the media.
One limitation of our study is that we are not able to better operationalise all
outputs from the ombudsman’s communications branch. We only have press
releases while the ombudsman’s office is in contact with several journalists, includ-
ing in an informal manner. In addition, when there is a press conference, the
ombudsman makes a speech (not available online and not included in our body
of texts), which can influence media coverage. It should also be mentioned that
cyclical events can also influence how the report is picked up by the media.
For example, if a report addresses a hot issue in the current affairs of Quebec,
the report could receive more media coverage. On the other hand, a high-profile
event can divert media attention and eclipse the presentation of a Québec
Ombudsman report. The data collected for this study do not allow us to examine
these phenomena. Further studies will be needed to answer these questions.
Moreover, in order to determine whether our results can be generalised, a future
study should compare the media coverage of publications from various ombuds-
men in the Canadian provinces or elsewhere in the world. As a less ambitious
approach, it would also be interesting to conduct a study that would analyse all
of the media coverage dealing with the ombudsman, as compared to our study,
which only covers articles dealing with the Québec Ombudsman’s reports.
Another limitation that we think needs to be mentioned is related to the quality
of media data. As noted in the methodology section, we used Eureka.cc. After a
recent update, Eureka.cc no longer allows articles to be saved in HTML and
requires them to be downloaded in unstructured PDF format, which makes con-
version to a database very difficult, if not impossible, for large volumes. Within the
framework of this article, Eureka.cc kindly agreed to provide us with the articles
corresponding to our research (see the set of conditions [keywords, dates and
sources] mentioned earlier) in HTML. However, when we do the search ourselves
in the Eureka.cc search engine, we do not get exactly the same number of articles,
which raises a problem about data-collection capabilities and the possible replica-
tion of the literature search.
360 International Review of Administrative Sciences 87(2)

In conclusion, we note that the existence of a press release issued by the Québec
Ombudsman appears to be a predictor of media coverage. The existence of this
press release seems not only to have a significant impact on the extent of media
coverage, but also to set the tone for the articles that follow. Finally, our data
allow us to suggest that the communication efforts made by the Québec
Ombudsman seem to have more impact than the reports’ actual tone.

Acknowledgements
We are grateful to the evaluators for their comments that improved this article, as well as the
contributions of Richard Dufour, Consulting Librarian, Antoine Baby-Bouchard and Anik
Dupuis, research assistants at the Public Policy Analysis Centre of Université Laval (CAPP).

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, author-
ship, and/or publication of this article.

Funding
The author(s) disclosed receipt of the following financial support for the research, author-
ship and/or publication of this article: We would like to thank the Québec Ombudsman and
specifically Mr Michel Clavet, the Fonds de recherche du Que´bec – Socie´te´ et Culture (FRQ-
SC) and the Research Chair on Democracy and Parliamentary Institutions at Université
Laval for funding this research.

ORCID iD
Dominic Duval https://orcid.org/0000-0003-0788-8631

Notes
1. ‘In the Québec parliamentary system, the appointed persons exercise, with a high degree of
autonomy, some of the powers of the National Assembly, powers that the Assembly could
otherwise assume itself’ (Québec National Assembly, 2015). In Québec, five persons are
appointed by the National Assembly. These are the Ethics and Professional Conduct
Commissioner, the Lobbyists Commissioner, the Chief Electoral Officer, the Public
Ombudsman and the Auditor General. They are appointed on a motion by the prime
minister, subject to the approval of two-thirds of the members of the National Assembly.
Most of the ombudspersons appointed since 1969 have had a legal qualification and a
background in the senior public service (eight out of nine) or the academic world.
2. Examples include ‘Compensation for victims of crime: For effective and prompt care of
vulnerable people’ (2016), ‘Home schooling: For better monitoring of children’s learning’
(2015) and ‘For services better adapted to prisoners with mental health problems’ (2011).
3. Examples include ‘Intervention report at the Centre intégré universitaire de santé et de
services sociaux de la Capitale-Nationale’ (2017) and ‘Intervention report at Hôpital
Maisonneuve-Rosemont’ (2017).
Duval et al. 361

References
Affodegon SW, Jacob S and Montigny É (2017) L’Ombudsman face aux défis de l’évalu-
ation: est-il possible d’évaluer l’intangible? Canadian Journal of Program Evaluation
32(2): 222–243.
Andersen R, Tilley J and Heath A (2005) Political knowledge and enlightened preferences:
Party choice through the electoral cycle. British Journal of Political Science 35(2): 285–302.
Ayeni V (1985) A typology of ombudsman institutions. Occasional Paper, Number 30,
Alberta, Canada, International Ombudsman Institute.
Bartlett C, Sterne J and Egger M (2002) What is newsworthy? Longitudinal study of the
reporting of medical research in two British newspapers. British Medical Journal
325(7355): 81–84.
Baudot P-Y and Revillard A (2011) Le médiateur de la République: périmètre et autonom-
isation d’une institution. Revue française d’administration publique 139: 339–352.
Benoit K, Watanabe K, Wang H, Nulty P, Obeng A, Müller S, Matsuo A (2018) Quanteda:
An R package for the quantitative analysis of textual data. Journal of Open Source
Software 3(30): 774. doi: 10.21105/joss.00774, https://quanteda.io.
Bernt W and Owen S (2000) The ombudsman in Canada. In: Roy G and Giddings PJ (eds)
Righting Wrongs: The Ombudsman in Six Continents. Amsterdam: IOS Press, pp. 127–142.
Binderkrantz A (2008) Different groups, different strategies: How interest groups pursue
their political ambitions. Scandinavian Political Studies 31(2): 173–200.
Brothers LM (2014) Identity and culture in ombudsman practice. Conflict Resolution
Quarterly 31(4): 421–434.
Cantelli F, Jacob S, Genard J-L, et al. (2006) Les constructions de l’action publique.
Paris: L’Harmattan.
Dupuy C and Pollard J (2012) À quoi servent les rapports administratifs d’expertise? Retour
sur une source e´crite centrale en sociologie de l’action publique. Travaux de science polit-
ique/Political Science Working Paper Series. Grenoble: PACTE.
Duval D and Pétry F (2016) L’analyse automatisée du ton médiatique: construction et
utilisation de la version française du Lexicoder Sentiment Dictionary. Canadian
Journal of Political Science/Revue canadienne de science politique 49(2): 197–220.
Farnsworth SJ and Lichter SR (2007) The Nightly News Nightmare: Television’s Coverage of
US Presidential Elections, 1988–2004. Rowman & Littlefield.
Fortier I (2009) Quebec’s ombudsman evolution. In: Stuart H (ed.) Provincial and Territorial
Ombudsman Offices in Canada. Toronto: Toronto University Press, pp. 211–240.
Fournel J-L (2007) Introduction à la ‘forme-rapport’: caractéristiques et temporalités d’une
production de vérité publique. Cultures et Conflits 65: 37–49.
Gayon V (2016) Écrire, prescrire, proscrire. Notes pour une sociogénétique de l’écrit bureau-
cratique. Actes de la recherche en sciences sociales 213(3): 84–103.
Giddings P (2000) The United Kingdom health service commissioner schemes. In: Roy G
and Giddings PJ (eds) Righting Wrongs: The Ombudsman in Six Continents, Amsterdam:
IOS Press, pp. 339–354.
Giddings P, Sladecek V and Diez Bueso L (2011) The ombudsman and human rights. In:
Roy G and Giddings PJ (eds) Righting Wrongs: The Ombudsman in Six Continents.
Amsterdam: IOS Press, pp. 441–457.
Gregory R and Giddings PJ (2000) The ombudsman institution: Growth and development.
In: Roy G and Giddings PJ (eds) Righting Wrongs: The Ombudsman in Six Continents,
Amsterdam: IOS Press, pp. 1–20.
362 International Review of Administrative Sciences 87(2)

Laurent D and Sanson M (eds) (1996) Le travail gouvernemental. Rapports de se´minaires

e´tablis par les e´le`ves de la promotion Rene´ Char. Paris: La Documentation française.
Lezertua M (2018) The twenty-first century ombudsperson: A guarantor of democracy. In:
Hertogh M and Kirkham R (eds) Research Handbook on the Ombudsman. Cheltenham:
Edward Elgar, pp. 474–484.
Maingueneau D (2002) Les rapports des organisations internationales: un discours consti-
tuant? In: Rist G (ed.) Les mots du pouvoir. Sens et non-sens de la rhe´torique internatio-
nale. Genève: Graduate Institute Publication, pp. 119–132.
Monogioudis G (2015) Tracing public accountability in Serbia: The ombudsman institu-
tions in search of allies. Doctoral thesis, University College London, UK.
Mordicai L (2005) Public reporting. In: Rabin J (ed.) Encyclopedia of Public Administration
and Public Policy. Middletown, PA: Taylor & Francis, pp. 239–243.
Patterson TE (1993) Out of Order. New York, NY: Knopf Inc.
Penissat E (2012) Publier des ‘chiffres officiels’ ou les contraintes bureaucratiques et polit-
iques qui façonnent l’expertise d’État. Gouvernement et action publique 4(4): 45–66.
Pomper GM and Lederman SS (1980) Elections in America: Control and Influence in
Democratic Politics (2nd edn). New York, NY: Longman.
Québec National Assembly (2015) Personne désignée. Encyclope´die du parlementarisme que´-
be´cois (online).
Reif LC (2004) The Ombudsman, Good Governance, and the International Human Rights
System. Leiden: Martinus Nijhoff Publishers.
Simonian-Gineste H (2015) Du Médiateur de la République au Défenseur des droits. Revue
Miroirs 2: 66–85.
Soroka SN (2014) Negativity in Democratic Politics: Causes and Consequences. Cambridge:
Cambridge University Press.
Truan N and Renard L (2017) Dire l’intégration. Les rapports français et allemands sur
l’intégration entre injonction à la neutralité et volonté d’influence (1991–2014). Mots. Les
langages du politique 114(2): 77–93.
Walgrave S and De Swert K (2007) Where does issue ownership come from? From the party
or from the media? Issue–party identification in Belgium, 1991–2005. Press/Politics
12(1): 37–67.
Wing-Hung Lo C, Wai Hang Yee H, Ning Liu, N et al. (2011) Ombudsman and stakeholder
engagement for improved service delivery. In: Carmona GV, Waseem M, Brillantes AB,
Ranjan JR, Tiu Sonco II JO, Wing-Hung Lo C, Li H, Ning Liu N, Hang Yee W, Martin
A and Jones G. (eds), Strengthening the Ombudsman Institution in Asia: Improving
Accountability in Public Service Delivery through the Ombudsman. Manila: Asian
Development Bank, Collection ‘Public Management, Financial Sector, and Trade’, pp.
117–189.
Young L and Soroka S (2012) Affective news: The automated coding of sentiment in polit-
ical texts. Political Communication 29(2): 205–231.

Dominic Duval is Professor at the Department of Social and Public

Communication at the Université du Québec in Montréal, Canada. His research
focuses on election promises, political communication and public opinion.

Steve Jacob is Full Professor and Director of the Centre for Public Policy Analysis
(CAPP) at Université Laval, Canada. He holds the Research Chair in Public
Duval et al. 363

Administration in the Digital Age. He conducts research on modernising administra-

tive processes, public ethics, and evaluation systems and performance management.

Eric Montigny is Professor in the Department of Political Science at Université

Laval, Canada, Scientific Director of the Research Chair on Democracy and
Parliamentary Institutions, and an associate researcher with the Research Group
on Political Communication and the Institute of Applied Ethics. His research
focuses on the evolution of parliamentarianism, activism and relationships based
on common interest.

Mathieu Ouimet is Full Professor in the Department of Political Science at

Université Laval, Canada. He is a researcher at the Research Chair on
Democracy and Parliamentary Institutions, the Centre for Public Policy
Analysis (CAPP) and the Research Centre of Québec University Hospital
Centre. His research focuses on the use of scientific knowledge in public adminis-
tration and lobbying.