Risk-based regulatory delivery:

Considerations for implementation

By Patricia Larkin, PhD

Prepared for the

Community of Practice in Risk-Based Regulatory Delivery
Community of Federal Regulators

Ottawa, Canada
April 28, 2018
Contents
Executive Summary ........................................................................................................................ 3
Section 1: Introduction ................................................................................................................... 6
a. Risk-based regulatory delivery in brief................................................................................ 6
b. Report outline....................................................................................................................... 8
c. Risk assessment and management process ........................................................................ 10
Section 2: Considerations for risk assessment .............................................................................. 17
a. Risk profiles ....................................................................................................................... 17
b. Data and methods ............................................................................................................... 19
c. Risk control options analysis - connection to risk management phase .............................. 22
Section 3: Considerations for risk management .......................................................................... 26
a. Compliance tools and enforcement mechanisms ............................................................... 26
Prioritizing the approach ....................................................................................................... 29
b. Measurement, monitoring and verification ........................................................................ 31
Section 4: Considerations for risk communication ...................................................................... 34
a. Stakeholder engagement .................................................................................................... 34
Third parties .......................................................................................................................... 35
b. Open and transparent processes and outcomes .................................................................. 36
Section 5: Other support for risk-based regulation and risk-based regulatory delivery .............. 38
a. Leadership .......................................................................................................................... 38
b. Training .............................................................................................................................. 39
c. Evaluation, review and adjustment .................................................................................... 40
References ..................................................................................................................................... 42
Appendices .................................................................................................................................... 46
Appendix 1 - Conceptual Frameworks ..................................................................................... 47
Appendix 2 - Glossary .............................................................................................................. 56
Appendix 3 - Risk metrics, Risk ranking .................................................................................. 62
Appendix 4 - Potential tools for low risk sites or activities ...................................................... 64

2
 Executive Summary

This report was prepared in support of the Community of Practice (CoP) for Risk-Based Regulatory
Delivery launched by the Community of Federal Regulators in December 2017. While risk-based
regulation (RBR) refers to a regulatory technique writ large, this report provides an overview of
considerations for implementation (including enforcement) within the public policy cycle, including
pertinent issues for oversight (regulatory review). The implementation step within RBR is
hereinafter referred to as risk-based regulatory delivery (RBRD) in order to match the name given to
the CoP.

It is understood readers have a range of experiences in RBR and RBRD on which to build. For some,
the considerations presented here will provide a solid background; for others, this report could
serve as a review. For those further along the continuum, the discussion may spur enhanced
implementation.

Fundamental components of RBRD are to:

 apply principles of risk assessment to the experience and drivers of the sector or regulated
entity, and hazards themselves
 target regulatory resources based on the degree of risk the regulated entity(s) poses to the
regulator’s public policy objectives
 apply a range of risk management (compliance and enforcement) options in proportion to
the probability and severity of the consequence of non-compliance
 engage in open and transparent risk communication during the decision-making process.

As an approach to identify common considerations in risk assessment, risk management, and risk
communications, seven international frameworks were reviewed:
 European Aviation Safety Agency’s (EASA) practices for risk-based oversight
 Australia’s risk management approach to the regulation of therapeutic goods
 New South Wales (NSW), Australia, guidance for regulators to implement outcomes and
risk-based regulation
 Canada’s financial sector’s risk-based approach to combatting money laundering and
terrorist financing
 Alberta Energy Regulator’s (AER) integrated compliance assurance framework
 BC Oil and Gas Commission’s (BCOGC) compliance protocol for facilities management

Several of these guidance documents are directed at regulated entities as opposed to support for the
regulators’ risk-based decision-making process. Apparently the latter remain less accessible. An
example of this was the author’s attempt to obtain the British Columbia Oil and Gas Commission’s
risk-based criteria to select permit holders who must participate in the compliance assurance
process. No response was received. Nevertheless, the regulated entity’s guidance was included in
this review because considerations for risk assessment, risk management and risk communication
were found to be similar to those discussed in documentation developed for a regulator, such as the
New South Wales guidance document. Additional review of the grey literature and academic
journal articles also form the basis of the discussion.

Findings
In the risk assessment phase of RBRD, two major steps include risk dimension analysis and risk
estimation. Risk profiles are created at the sector level or for an individual regulated entity. An
activity risk profile, perhaps the best recognized application of risk assessment and management, is

3
also completed. A key consideration is for regulators to understand and attempt to change the
motivations of regulated entities to (voluntarily) comply with the law, as well to identify factors
that lead to non-compliant behaviour. The focus is on the cause and not just the symptoms.
Evidence-informed RBRD needs data that is relevant, timely, objective, best available, and at the
most reasonable cost. However, data gaps and compatibility issues for both leading and lagging
indicator sets are ongoing concerns. A third step in risk assessment, as discussed in the report, is
risk control options analysis that links results from the contextual analysis and risk estimation with
the risk management phase. There may be a level of risk below which regulators will not consider
certain enforcement responses, and above which regulators will start to consider more serious
enforcement responses. Furthermore, compliance and enforcement for low risk businesses and
activities form the bulk of the regulated population and this group could be further categorized to
assist with resource allocations. Special attention is drawn to low probability/high impact
circumstances because a poor outcome could not only affect human health and the environment
(and other risk issue areas), but also public perception and public acceptance of RBR/RBRD overall.
Selected tensions in the risk assessment phase include data availability and agreement on criteria
used to determine risk tolerance/risk acceptability.

With respect to risk management, major RBRD steps include the decision for which tool or strategy
to implement, the sequence, and monitoring results. Risk management options that may influence
compliance and non-compliant findings include those of a regulatory, economic, advisory,
community-based, or technological nature. Soft and hard/strong mechanisms are presented. The
mission of regulators is to maximize compliance levels, potentially using criteria and/or thresholds
to determine appropriate risk management strategies. In addition to the use of indicators and
metrics in risk assessment, a critical issue for risk management is how to measure and monitor
compliance in terms of general deterrence and effectiveness. Continual improvements in
monitoring and reporting are the mechanics through which the benefits of RBRD are capitalised
and maximised over time. Decisions for monitoring frequency are made in order to be in a position
to identify trends and also to feed into a regulatory review (oversight) process. Tension in the risk
management phase includes the extent to which a regulator can apply contextual discretion while
exhibiting a consistent compliance and enforcement approach.

Considerations for risk communication are also provided. Meaningful stakeholder engagement and
transparency (defined in terms of ease of access and the fullest possible disclosure) is emphasized
for the risk assessment and management process as well as the regulators’ compliance and
enforcement decisions. Considered best practice, a list of stakeholders might include internal,
external, community, practitioner, industry, multi-levels of government, and international
regulatory bodies. A sub-section of risk communication identifies a range of third parties that may
have a role in RBRD, regulatory review, and RBR more generally. The importance of engagement,
including recognition of both the regulators’ and public values, cannot be overstated as a hallmark
of an advanced level of democratization. Tensions in risk communication activities include the
perceived relationship that may develop between regulators and regulated entities as well as
ensuring open and transparent risk communication, which is the least developed component of the
frameworks reviewed here. An effort to identify ways to enhance and embed communication
activities within RBRD is recommended.

This report concludes with considerations that provide overarching support for RBRD and RBR in
general. Leadership and training will underpin effective implementation. Moreover, ongoing
evaluation, review and adjustment should be used in an effort to identify gaps and limitations in the
regulators’ approach, thereby linking back to the place of RBRD within the broader RBR policy
cycle, particularly with respect to oversight (regulatory review). The continuum of compliance and

4
enforcement does not always address risk issues adequately and new or adjusted efforts become
necessary. In this process, open and transparent engagement with internal and external
stakeholders is again required.

In summary, preconditions for success include reliable assessment of the probability and
consequences of adverse outcomes; clarity and agreement for the goals and trade-offs, including
political tolerance for adverse outcomes; access to a range of enforcement tools that can be
deployed in proportion to the context (severity, drivers, costs, and benefits); and open and
transparent decision-making processes and outcomes.

5
 Section 1: Introduction

The dual objective of any jurisdiction’s public policy cycle is to “improve the welfare of citizens by
providing better protection from hazards and more efficient services from government” (OECD,
2010, p. 15). As a regulatory technique, risk-based regulation (RBR) is highlighted as an effective
approach to give priority to serious and important matters (Baldwin and Black, 2016).
Australia’s New South Wales Government (2016) suggested this approach can enhance collective
economic and social wellbeing by reducing unnecessary regulatory burden on regulated entities;
increase the productivity of regulators and regulated entities; and drive flow-on economic and
social benefits.

This report has been prepared for the Community of Federal Regulators’ Community of Practice
(CoP) for Risk-based Regulatory Delivery, recognizing the complexity, variability, multi-sources of
hazards, and technological change that contribute to a dynamic environment in which to address
risk issues. With renewed interest in advancing RBR in the Canadian context, this report considers
important issues for its application, with particular attention to implementation (including
enforcement) within the public policy cycle1, hereinafter referred to as risk-based regulatory delivery
(RBRD) to mirror the name of the CoP. Indeed, Coglianese (2015) concluded that ‘best in class’
regulators should facilitate effective deployment of scarce resources; improve consistency; provide
evidence that risks are not being over- or under-regulated; and encourage compliance with
legislation.

It is understood readers have a range of experiences in RBR and RBRD on which to build. For some,
the considerations presented here will provide a solid background; for others, this report could
serve as a review. For those further along the continuum, the discussion may spur enhanced
implementation, perhaps with a focus on risk communication.

Section 1 provides a brief overview of RBRD, the research methodology, and components of a
generic risk assessment (RA) and risk management (RM) process. Sections 2, 3, and 4 describe
considerations related to the RA, RM, and risk communications activities. Discussion includes RBR
more generally and oversight (regulatory review), as appropriate. For example, stakeholder
engagement and open and transparent processes are key to developing overall trust, credibility and
support for this regulatory technique. Section 5 completes the report, highlighting support for RBR
and oversight functions through leadership and training, and the public policy step to evaluate,
review, and adjust.

a. Risk-based regulatory delivery in brief

Members of the CoP may represent a wide variety of provincial ministries or federal departments
engaged in regulation. Major categories2 include protection for health and environmental risks;
markets, banking, and insurance; labour and professions; public safety and security; family life and
child protection; and animal welfare. At the federal level, risk management is embedded as a
critical activity, with guidance related to accountability, transparency, and risk-informed decision-

1
The public policy cycle typically includes six major steps (OECD, 2010): Forecasting (risk
assessment); Prevention (risk management); Oversight (regulatory review); Implementation
(including enforcement); Coping (adaptation, remediation, crisis response/disaster relief, and
compensation, if prevention fails and a risk comes to pass); and Evaluation.
2
Proposed by Leiss, 2003, unpublished. Smart Regulation and Risk Management, prepared for Privy Council
Office

6
making provided in the Framework for the Management of Risk (Government of Canada, 2010b) and
Framework for the Management of Compliance (Government of Canada, 2010a). Similarly, in the
United Kingdom where RBR has been implemented for about a decade, the Regulators’ Code (UK
Better Regulation Delivery Office, 2014) outlines requirements for authorities to support
compliance and growth, base regulatory activities on risk, share information about compliance and
risk, and be transparent.

Fundamental components of RBRD are to:

 apply principles of risk assessment to the experience and drivers of the sector or regulated
entity, as well as to the hazards themselves
 target regulatory resources based on the degree of risk the regulated entity(s) poses to the
regulator’s public policy objectives
 apply a range of risk management (compliance and enforcement) options in proportion to
the probability and severity of the consequence of non-compliance with regulatory goals
 engage in open and transparent risk communication to develop trust and credibility for the
decision-making process.

Operationally, RBRD can be applied at various scales. At the sector level, a risk profile is completed
in order to develop a deep understanding of the effects of risks that sector participants must
address (European Aviation Safety Agency, 2016). Similarly, the Alberta Energy Regulator (AER)
(2016b) suggested that when combined and analyzed, the data collected from verification activities
provides information about compliance rates for a sector, regulated entities, or geographical area;
and that this information is essential in compliance assurance program planning and management.

RBRD is not static. The Public Risk Management (PRISM) Institute (2018 (unpublished)) suggested
an effective program is dependent on policies that consider institutional infrastructure, account for
stakeholder input, and provide the necessary flexibility, tools and instruments for
operationalization. Black and Baldwin (2012) noted that for low risk environments (although this
can also be said for all risk environments) this approach needs to be operable, dynamic,
transparent, and justifiable.

Criteria for RBRD success concern both organizational and process issue areas. Beaussier et al.
(2016, p. 205) concluded that several preconditions exist for successful implementation:
 “goals must be clear and trade-offs between them amenable to agreement;
 regulators must be able to reliably assess the probability and consequences of adverse
outcomes;
 regulators must have a range of enforcement tools that can be deployed in proportion to
risk; and
 there must be political tolerance for adverse outcomes”.

On this last point, the purpose of RBR (and therefore RBRD) is not to eliminate risk (which is not
possible, as there is no such thing as ‘zero-risk’). Regulators and politicians are also required to
take risks (Black, 2008) and in implementing this approach, public and political tolerance for
adverse outcomes needs to be cultivated and described (BC Oil and Gas Commission, 2017). Early,
ongoing, open, and transparent communication amongst stakeholders could assist with identifying
and agreeing on tolerable risk levels.

7
b. Report outline
This report considers common RBRD (implementation) elements in seven frameworks,
representing international, national, and provincial jurisdictions (Table 1, graphical representation
in Appendix 1). Additional reference is made to the grey literature and academic journal articles.

1. European Aviation Safety Agency’s (EASA) practices for risk-based oversight (European
Aviation Safety Agency, 2016)
EASA collected risk-based oversight practices from 13 European states that agreed to share their
experience. The document elaborates on the assessment models for organisation risk profiles and
oversight planning/execution. It also discusses the tools and enablers available for continuous
review of the risk profile considering safety performance and management of change policies.

2. Australia’s risk management approach to the regulation of therapeutic goods (Australian

Government, 2011, 2013)
Australia’s Therapeutic Goods Administration (TGA) monitors the quality, safety and performance
of therapeutic goods when they become available to consumers to ensure the on-going compliance
of the products with TGA’s regulatory requirements. There is an on-going program of to verify the
suitability of manufacturers to produce therapeutic goods for supply in Australia.

3. Australia’s Office of the National Rail Safety Regulator’s (ONRSR) framework for risk-based
regulation, compliance and enforcement policy (Office of the National Rail Safety Regulator, 2016)
The Rail Safety National Law is based on the principle of shared responsibility for safety on all
parties, including rail transport operators, rail safety workers, other persons and duty holders
involved in the rail industry, and the public. The Office of the National Rail Safety Regulator has the
principal objective of facilitating the safe operation of rail transport services across Australia.

4. New South Wales (NSW), Australia, guidance for regulators to implement outcomes and
risk-based regulation (NSW Government, 2016)
NSW combines an outcomes and risk-based approach to regulation, in the context of having to
deliver more with fewer resources. Outcomes and risk-based regulation provides regulators with a
consistent and transparent framework to proactively respond to that challenge, while also
increasing their effectiveness in achieving regulatory outcomes using a contribution analysis. The
guidance includes a diagnostic tool for regulators to identify their level of maturity in
implementation (NSW Government, 2018).

5. Canada’s financial sector’s risk-based approach to combatting money laundering and

terrorist financing (Government of Canada, 2017b)
Three cornerstones of this compliance program are for regulated entities to develop policies and
procedures, complete a risk assessment of activities, and demonstrate risk mitigation measures. A
two-year effectiveness review of activities is mandatory.

6. Alberta Energy Regulator’s (AER) Integrated Compliance Assurance Framework (Alberta

Energy Regulator, 2016a, Alberta Energy Regulator, 2016b)
Two strategic goals of the AER for RBRD are: to be effective, by strengthening risk-based regulation
such that regulated entities are aware and understand their requirements, all while the AER
supports changes in behaviour; and to be credible, by increasing trust and confidence, including
compliance activities that are fair, transparent, and equitable, with clear understanding of roles and
responsibilities.

8
7. BC Oil and Gas Commission’s (BCOGC) compliance protocol for facilities management (BC
Oil and Gas Commission, 2017)
The purpose of this protocol is to provide guidance to facility owners and operators outlining the
Commission’s requirements and expectations with regards to developing, implementing and
maintaining an integrity management program for facilities (IMPF). All regulated facility permit
holders are required to participate in the IMPF compliance assurance process. The Commission
uses a risk-based selection process to identify permit holders to participate in an IMPF compliance
assurance process.

New South Wales - Guidance

BC Oil and Gas Commission

Alberta Energy Regulator
International – European

Canada - Financial sector

Australia - National Rail
Australia – Therapeutic
Aviation Safety Agency

money laundering and

Facilities Management
Goods Administration

terrorist financing
Guidance inclusions

for regulators
Safety

Principal audience3
Risk assessment (RA)
RA Sector profile
RA Entity profile
RA Activity/ Performance
Sharing information
Evaluation of risk tolerance
Compliance and
enforcement options
Monitoring, Verification
Leadership, training
Program evaluation
Stakeholder engagement
Transparency
Link to strategic
management
R P R R, P R R R
X4 X X X Y X
X Y X X
X Y Y X X5 X X
X X Y X X X X
X Y Y Y
Y X X X
Y Y6 X Y Y X Y
X X X X X Y X
X Y Y X Y X
X X X Y Y X
Y X X Y7 Y8
Y X X X
Y X X X X Y

Table 1: Selected inclusions in seven risk-based regulatory delivery frameworks

3
P - public; R - regulated entities
4
X - elaborated; Y - identified
5
Guidance for different entity types
6
Detailed in therapeutic goods legislation
7
Detailed in AER’s Stakeholder Engagement Framework - https://www.aer.ca/documents/about-
us/StakeholderEngagement_Framework.pdf
8
Detailed in BCOGC Application Manual, Chapter 6 - https://www.bcogc.ca/node/13267/download

9
Criteria used to choose the frameworks includes their accessibility, elaboration, implementation
status, jurisdiction, and sectoral domains. Published guidance for regulators and for regulated
entities was analysed based on the similarity in the key risk assessment, risk management, and risk
communication elements for both audience types. It was found that guidance for regulators is less
available. An example of this was the author’s attempt to obtain the British Columbia Oil and Gas
Commission’s risk-based criteria to select permit holders who must participate in the compliance
assurance process. No response was received. Additional review of the grey literature and
academic journal articles also form the basis of the discussion.

Considerations include:
Section 2: Risk assessment - risk profiles; data and methods; risk control options analysis
Section 3: Risk management - compliance tools and enforcement mechanisms, including
prioritizing the approach; measurement, monitoring and verification
Section 4: Risk communication - stakeholder engagement, including third parties;
transparency in process and outcomes
Section 5: Support for RBR and RBRD - leadership, training, and program evaluation.

A glossary of common terms and definitions is also provided for reference purposes (Appendix 2).

As noted, this review is focused on considerations for the implementation step (including
enforcement) within the public policy cycle. A risk assessment and management process regarding
the need for, design, or development of new state-led regulations is not discussed. Section 5,
however, includes consideration of the importance of program evaluation to support the public
policy cycle, thereby linking back to risk-based regulation technique overall. Furthermore, specific
methods and techniques for risk assessment are not described; however a range of compliance and
enforcement risk management options is provided. Finally, an examination of jurisdictional
practice with respect to the chosen frameworks was not completed. A subsequent report could
consider the outcomes of regulatory practice - for instance, to assess whether applying the steps in
the assessed frameworks has enabled the regulator to:
 target inspections in such a manner as to maximize the chance of finding and reducing
significant regulatory violations; and/or
 deploy enforcement tools responsively, calibrating consequences so as to assure
compliance and promote positive cooperation.

A brief description of the risk assessment and management process is provided next, given this is
fundamental to RBRD (implementation).

c. Risk assessment and management process

Risk is a concept that integrates the likelihood and severity of the consequence of adverse outcomes,
whether to health, the environment, the economy, politics, or even reputation (See also Glossary).
There are numerous recognized methodologies for how such problems are explored, but risk
assessment and risk management (RA/RM) are undertaken conjointly and are fundamentally
matters of judgement in probability and uncertainty (National Research Council, 1983).

Potential adverse outcomes (harm) may affect individuals or populations (often vulnerable
populations) as a result of hazards inherent in the culture or activities of organizations subject to
regulation. RA/RM needs to be a well-developed, documented, and justifiable process where the
regulator identifies, rates and finds ways to mitigate the risk of hazards based on an understanding
of acceptable societal risk and risk control options. Trade-offs are unavoidable. Government,

10
industry, 3rd party, and general public stakeholders should be engaged throughout a systematic,
multi-step, open and transparent process. A generic depiction of the RA/RM process is provided
(Table 2).

Major Step Key components

Risk Assessment Phase
1. Ongoing surveillance a. Horizon scanning
b. Risk forecasting/foresight
c. Science updates
2. Review policy/governance a. Domestic laws, regulations
context b. International treaties and
agreements
Stakeholder Engagement, open and transparent Risk Communication

c. Relevant policies
3. Risk dimensions analysis a. Number of unique risks
b. Potential for risk escalation
c. Initial risk communication
4. Risk estimation a. Hazard characterization
b. Exposure characterization
c. Determination of risk factors
d. Frequency estimation
e. Consequences estimation
f. Uncertainties specification
g. Risk class (Probability x
Consequence matrix)
h. Risk ranking
5. Risk control options a. Domestic context
analysis b. International context
c. Financing of options
Risk Management Phase
6. Formal consultation a. Interim decisions (if required)
process b. Presentation of risk assessment
c. Analysis of feedback
7. Risk management decision a. Risk control steps
b. Allocation of responsibilities
c. Risk communication strategy
8. Implementation sequence a. Coordination of agencies
b. Memoranda of agreement
c. Targets and timeframes
9. Monitoring and compliance a. Allocation of responsibilities
plan b. Inspections and reporting
c. Achieving risk control objectives
10. Evaluation, review and a. Agreed timeline for follow-up to
adjustment verify predictions
i) steps for risk assessment
ii) steps for risk management

Table 2: Generic multi-step risk assessment and management process, including sub-components

11
A step-wise RA/RM process is useful at all levels of decision making. It is worth noting here that the
language used to describe the public policy cycle also includes risk assessment and risk
management as note previously. A regulator embarking on regulatory review, as a distinct activity,
therefore also follows the same essential process. While the details were not available at the time
of writing and the comment period has passed, the Underwriters Laboratories Inc. will publish a
Guideline for Managing Risks to the Public Interest in a Regulatory Context. This guidance is focused
on governance of public risk management and foundational principles and elements for a public
risk management framework which may also be of interest to members of the CoP.

As examples, a number of these steps are represented in elaborated graphics for three RA/RM
frameworks reviewed for this report. The Australian Therapeutic Goods Administration (TGA)
published guidance that explains their approach to a general audience (Australian Government,
2011) (Figure 1). As good practice, the TGA outlines 7 steps in a risk management process using
the generic AS/NZS ISO Standard 31000 that broadly matches Table 2. Steps 1-4 reflect
components in the risk assessment phase; steps 5-6 reflect risk management phase; and step 7
includes communication and consultation.
1. Establishing the context. For example, defining the relationship between the organisation
and its environment, understanding the organisation’s capabilities and identifying the
internal and external stakeholders of the organisation.
2. Risk Identification. This involves identifying the risks that need to be managed.
3. Risk Analysis. The objectives of analysis are to separate out the minor acceptable risks from
the major risks and to provide data to assist in the evaluation and treatment of risks. Risk
analysis involves consideration of the sources of risk, their consequences and the likelihood
that those consequences may occur.
4. Risk Evaluation. Risk evaluation involves comparing the level of risk found during the
analysis process with previously established risk criteria.
5. Risk Treatment. This involves identifying the range of options for treating risk, assessing
those options, preparing risk treatment plans and implementing them.
6. Monitoring and Review. It is necessary to monitor risks, the effectiveness of the risk
treatment plan, strategies and the management system which is set up to control
implementation. Risks and the effectiveness of control measures need to be monitored to
ensure changing circumstances do not alter priorities.
7. Communication and Consultation. This is an integral part of all aspects of the risk
management process.

12
Figure 1 - Australia’s Therapeutic Goods Administration risk management framework (AS/NZS ISO
Standard 31000)

13
The BC Oil and Gas Commission (BCOGC) (2017) (Figures 2-3) and Government of Canada (2017b)
(Figure 4) published elaborated guidance for the requirements and expectations for regulated
entities to develop, implement, document, and maintain their risk assessment and management
activities. BC’s lifecycle requirements and expectations for the integrity management program for
facilities are based on a management systems approach with 16 components within four broad
categories:
 Planning - Leadership Commitment; Risk Assessment and Management (with structured
approach)
 Implementing - Communication Process; Organizational Structure, Roles and
Responsibilities; Training and Competency; Document and Records Management; Managing
Change; and Operational controls
 Checking and Evaluating
o Risk Management - Inspection, Monitoring and Maintenance; Evaluation and
Fitness-for-Service Assessment; Modification and Repair
o Program Assessment and Improvement - Incident/Near-miss Investigation and
Learning; Control of Non-conformances; Internal Audits; Performance Measurement
and analysis of Data
 Act - Management Review

The BC framework is intended to be applied with “flexibility to account for existing internal
programs and processes that already cover issues relating to the compliance protocol for facilities
management” (BC Oil and Gas Commission, 2017, p. 8). The guidance serves as a basis to compare
and review the regulated entity’s (permit holder’s) programs and systems with those of the
Commission’s protocol.

Figure 2 - BCOGC Integrity Management Program for Facilities

14
Figure 3 - BCOGC risk management for the Integrity Management Program for Facilities

15
FINTRAC (Government of Canada, 2017b) provides a graphic representation of the six-step cycle
for regulated entities to complete in the assessment of products and services that are at risk of
money laundering and terrorist financing (Figure 4):

1. identification of inherent risks (business-based risk assessment along with the relationship-
based risk assessment);
2. setting risk tolerance;
3. creating risk-reduction measures and key controls;
4. evaluating residual risks;
5. implementing risk-based approach; and
6. reviewing risk-based approach.

Figure 4 - Guidance for risk assessment and management of money laundering and terrorist
financing

16
 Section 2: Considerations for risk assessment

Considerations for risk assessment include risk profiles; data and methods; and risk control options
analysis that marks the connection to the risk management phase.

a. Risk profiles
The risk assessment phase in RBRD (generically provided in section 1, Table 2) can be applied at
various scales: sector, regulated entity, and activity.

A sector-based risk profile provides the regulator with an understanding of the bigger picture of
risk and the potential implications of regulatory actions on entities operating within a value-chain
of related activities. This may be particularly useful where a regulator is just beginning to
implement RBRD. Risk categories may be created in order to classify and prioritise regulated
entities and behaviours (European Aviation Safety Agency, 2016). Relative risk can be identified,
based on the purpose and characteristics of the entities or activities, then supporting prioritization
for subsequent entity/activity risk assessments. Sector specific guidance (workbooks) have been
developed for the risk-based compliance program for money laundering and terrorist financing
(Government of Canada, 2017a).

However, specific elements of an organization’s activities evolve, such that operations move the
entity or parts of the operations to another category. This may occur with a change of business
model or new activities. Sector based risk profiles have also been linked to issues with data
because sectors can change the relative importance of data types.

A second scale of assessment is focused on the risk profile of the organization. A key consideration
is for regulators to understand and attempt to change the motivations of regulated entities to
(voluntarily) comply with the law, as well as factors that lead to non-compliant behaviour. The
focus is on the cause and not just the symptoms. A range of issues examined in risk assessment
might affect risk management (regulatory compliance), including skills and capabilities,
motivations, and external pressures. Past performance is deemed an indicator of future risk.

In taking account of the drivers and corporate culture, Gunningham (2015) and Borley and Page
(2016) suggest motivations include:
 fear of detection and punishment by government enforcement agents;
 fear of humiliation or disgrace in the eyes of family members or social peers (social license);
or
 an internalized sense of duty, that is, the desire to conform to internalized norms and beliefs
about the right thing to do.

The Therapeutic Goods Administration (Australian Government, 2013) characterize “Regulated

entity - attitude to compliance” (including management and compliance systems) (Figure 5) in four
major groups within the spectrum of organizational culture: Committed to doing the right thing;
Trying to do the right thing but don’t always succeed; Don’t want to comply but will if made to; and
Decision to be non-compliant. [The TGA’s identified approach to compliance fall within the
discussion, section 3].

17
Figure 5: Australia’s Therapeutic Goods Administration’s approach to regulated entity compliance
(Australian Government, 2013)

The European Aviation Safety Agency (2016) provides a sample questionnaire regarding hazards,
culture, and procedures, including questions for the risk profile and oversight planning, risk
management and assessment of the management system, degree of cooperation, enablers and tools.
With respect to procedures, the risk profile of the organization would include an assessment of the
effectiveness and maturity of the organisation’s management system and the extent to which it
reflects industry-based best management practices. In their research, Phan and Baird (2015) found
that organizations experienced higher levels of environmental performance where they had more
comprehensive environmental management systems. However, implementing or even achieving
industry standards may show conformance but does not necessarily presume compliance with
regulatory requirements.

Creating a risk profile of a regulated entity’s products and activities is perhaps the best known
application of RA/RM. According to the generic framework (Table 2, section 1), in addition to
completing the risk profile of the regulated entity, this would also document the risk estimation for
products or services within a risk register: hazard characterization, exposure assessment,
determination of risk factors, frequency estimation, uncertainty specifications, risk class
(probability × severity of consequence matrix), and risk ranking.

18
Overall, the risk profiles become information for risk management options analysis (discussed
below). Moreover, as suggested by the European Aviation Safety Agency (2016), the risk
perspective becomes an input to:
 the safety promotion process for more targeted actions and deliverables (section 3); and
 priority setting for the regulator, including internal training activities, interpretation or
update of existing rules, and initiation of new rules (section 5).

b. Data and methods

The first best practice principle suggested by the OECD (2014) is for regulatory enforcement and
inspections to be evidence-based and measurement-based: “deciding what to inspect and how
should be grounded on data and evidence, and results should be evaluated regularly” (p. 14).

In risk estimation (step 3, Table 2) and monitoring (step 9, section 3), RBRD needs data that is
relevant, timely, objective, best available, and at the most reasonable cost. Data gaps and
compatibility issues are ongoing concerns (Public Risk Management (PRISM) Institute, 2018
(unpublished)). BCOGC guidance notes that data availability is one of four most critical factors in
selecting risk assessment methods, along with “organizational maturity, goals, and the magnitude of
the decision associated with the risk analysis” (BC Oil and Gas Commission, 2017, p. 21).

OECD (2018) suggest specific objectives and indicators first be identified for high level goals, aims
or priority areas. Flexibility and possibly creativity appear to be important in addressing the need
for good data, a good system to collect data, and to target data collection based on different criteria
depending on the sector, entity and activity. The goal is to be proactive in gathering information
and identifying patterns, before an impact, or early on as problems develop in order to be in a
position to take action. Harms will be prevented. A brief discussion of risk metrics is provided
(Appendix 2).

Two types of indicators are (BC Oil and Gas Commission, 2017):
 Lagging indicators (outcome-oriented metrics) - the detection of events that have occurred;
data about incidents and failures; deficient performance of facility assets
 Leading indicators (process-oriented metrics or outputs) - how well various components
have been implemented and an indication of potential problems (including process,
program operating discipline, protective barriers to control risk and prevent incidents)

A list of data issues includes (Safety Management International Collaboration Group, 2013):
 validity, completeness, timeliness, access and availability, accuracy, absence of redundant
information; this includes legislative or operational restrictions on the use of data
 collection planning, standardization, structure and format (integration/fusion), collection
tools – ease of access, reporting; some of this relates to limited financial or human resources
allocations
 storage and database management, protection guidance (anonymity), and sharing.

Data sources can be varied, including sources from multi-level jurisdictions in addition to industry
itself. Data may be reportable by regulation, voluntary, observational (outliers from normal
operations), surveillance (audits, surveys, inspections to check for conformance), and screening.
While good reliable data may come from working with partners, with perceived value in
information sharing (OECD, 2018), there are potential limitations on the control of data collection,
especially where systems are controlled externally to regulators. Competent authorities should

19
develop arrangements for exchange of collected information and RBRD experiences, along with
potential cooperative actions on oversight (European Aviation Safety Agency, 2016, OECD, 2018).

With respect to the range of data available, while referring to the aviation sector, EASA (2016, p.
30) also suggested that “regulators should carefully govern the use of data to match the scope and
the nature of their role, tailoring their data feeds according to the size, maturity and complexity of
the aviation system they regulate. It is unlikely that a small aviation authority would have the same
data volume, granularity, resource, capacity or IT system sophistication as a large one”. This is true
of all sectors.

NSW Government (2016) provide a flowchart indicating data barriers and potential outcomes
(Figure 6). In beginning RBRD, this guidance suggested that a regulator look for “Quick wins”
because complete information/implementation will take time. The European Aviation Safety
Agency (2016) also discussed the needed timeline for data to grow and mature in order to add
necessary value. The UK Aviation Administration suggested this could take about 3 years.

In turning to risk assessment methods, these may vary between two timelines of risk
characteristics. For instance, NSW Government (2016) suggested a focus on quantitative analysis
for historical and current risks, in order to monitor forward indicator changes, to respond to
changing circumstances, and to test the impact of changes in assumptions with sensitivity analysis.
With a focus on transport safety, Nisula (2015) proposed creating a risk picture, with each
individual event being risk-assessed using scenarios for potential escalation of the outcome.

Emerging risk characteristics require “continual and coordinated scanning and brainstorming in
collaboration with other regulatory agencies, with clear assigned accountability” (NSW
Government, 2016, p. 20). There may be an emphasis on qualitative rather than quantitative
analysis because of the lack of data and experience.

Modeling results may be affected by data that is not available or not taken into account.
Gunningham (2015, p. 12) referred to Black and Baldwin (2012) in discussing the possibility “that
regulatory officials become committed to a historically captured set of risk indicators and
assessment criteria” that inhibit the regulator from taking account of data not captured by the
model, referring to this as “model myopia”. Similarly, European Aviation Safety Agency (2016)
suggested that models may not capture all the relevant risks; or that risk assessment may not be
adequate thus delivering misleading warnings. Accordingly an excellent regulator needs to be
constantly self-evaluating, learning and adapting its approach, identifying emerging problems and
acting promptly when it does so. Emerging risk issues may therefore benefit from technical and
subject matter expertise and possibly a variety of forums in order to obtain additional points of
view (section 4).

Overall, the NSW Government (2016) suggested keeping the risk assessment simple, without
eliminating important detail, with the view that this can support more effective outcomes including
improved communication with staff and the ability to monitor consistency. Additional guidance on
how to keep the risk assessment simple was not provided. The jurisdictional practice with respect
to this recommendation needs to be investigated further.

A discussion of data and methods would be incomplete without dealing with the issue of
uncertainty in risk assessment. Barlow et al. (2015), with respect to ensuring food safety, identified
the challenge to better understand uncertainties in risk assessment: how they vary in type and in
impact on a risk assessment, and how they should be interpreted. As discussed in section 4, there is

20
a need to be transparent about uncertainties in risk assessments and to explain uncertainties to all
stakeholders.

Figure 6: Potential barriers, implications and outcomes associated with data and systems capability
(NSW Government, 2016)

21
c. Risk control options analysis - connection to risk management phase
The last step in the generic risk assessment phase (step 5, Table 2) is to identify options for how
hazards and related risk issues can be reduced and managed to an acceptable/tolerable level, that
is, to determine risk control (mitigation) requirements and options. Notably, criteria at this stage
can be used to guide prioritisation of compliance and enforcement actions. For instance, Australia’s
Therapeutic Goods Administration (Australian Government, 2011, p. 9) focuses on
 Issues that may have adverse health consequences for consumers as a result of public
access to inappropriate or dangerous goods;
 Issues that may affect TGA's reputation among key stakeholders leading to a loss of
confidence in the TGA's regulatory processes and subsequently loss of confidence in
available therapeutic goods.

Results from the risk profile analysis, based on quantitative and qualitative data analysis as well as
the regulator’s judgement, can be represented in a risk heat matrix (Figure 7). A qualitative
description is provided for each level of “likelihood of non-compliance” and “risk or harm to health
and environment”. [As discussed in section 3, a heat matrix can also identify the compliance and
enforcement option best suited to the circumstance].

Figure 7: Generic risk heat map (Environment Protection Authority Victoria, 2017)

The federal guidance on the risk-based approach to combatting money laundering and terrorist
financing (Government of Canada, 2017b) includes a heat matrix, with a depiction for the level of
resources required within each unit (on a scale of 0 through 3): action (i.e. the need to respond to
risk), effort (i.e. level of effort required to mitigate the risk), and monitoring (i.e. level of monitoring
required).

22
Risks are ranked (overview, Appendix 3). However, there may not always be a match in the risk
ranking process between the results of a stakeholder consultation process and the prioritization of
risk reduction activities; and this may lead to a loss of trust with respect to participation in the
process (Paoli and Wiles, 2015) (section 4).

Risk control guidelines to support decision-making can be developed. A Transport Canada

framework for enforcement options includes ratings factors to help characterize the offence and the
offender characteristics (Government of Canada, 2018) (Figure 8). Risk mitigation is
commensurate with the risk tolerance (or acceptability) identified in setting the context for the
sector, regulated entity, or activity, and “the [residual] risks are not greater than what you are
prepared to tolerate” (Government of Canada, 2017b). NSW Government (2016) suggested there is
a certain level of risk that:
 “below which, regulators may not consider certain enforcement responses, and
 above which, regulators will start to consider more serious enforcement responses” (p. 48).

Figure 8: Graduated Approach - Determining factors in the offence and offender (Government of
Canada, 2018)

In the rail sector, as an international example, “across the various regulatory jurisdictions (world
over), a consensus is necessary to establish transportation risk acceptability criteria”9.

Finally, once the sector/entity profile and activity risks have been ranked, priorities are set, and this
step can affect the distribution of resources (Paoli and Wiles, 2015). The regulator therefore needs
further information on the costs, benefits, and feasibilities of interventions.

Importantly,
“These estimates may be as uncertain as the risk estimates are, and may add further
complexity to the social process, but the alternative is either to rank the risks alone and
have no guide for policy, or (perhaps worse) for decision makers to assume that the risk
ranking equals the resource allocation” (Finkel, 1996 in (Paoli and Wiles, 2015, p. 27).

9
Dr. Phani Raj - Hazardous Materials - US Federal Railroad Administration

23
With an emphasis in RBRD for risk control of high risk entities and activities, an additional issue in
options analysis concerns two other categories: routine (low risk) and low probability/high impact
(LPHI). Each of these elicits brief consideration.

Black and Baldwin (2012) discussed the issue of compliance and enforcement for low-risk
circumstances that form the bulk of the regulated population. Similarly, and related to the above
mentioned issue of acceptable risk/risk tolerance, Daku10 discussed how a regulator could identify
a minimum frequency or “risk floor” in prioritizing enforcement activities. Indeed, ‘low risk’ often
means ‘low priority’ in terms of relative significance and attendant decision-making with respect to
resourcing. Given the number of sites or activities that are categorized as such, Black and Baldwin
(2012) ask: is it possible and/or useful to further break the ‘low risk’ category to devise regulatory
strategies? Considerations here might include the extent to which the risk is stable or volatile and
the review period of the control measures11. Additional issues include the assessment process, in
bundling sites or activities (thereby obscuring some risks), and in fact, the resources needed to
undertake the categorization itself. Table 3 describes types of low risks. Black and Baldwin (2012)
also discuss regulators’ responses to low risk and potential tools for low-risk sites/activities
(Appendix 3).

With respect to LPHI, dedicated risk assessment, risk control options analysis, and risk
management is necessary to both support health and wellbeing and ensure continued public trust
and credibility for the regulator. As suggested by Sparrow (2008), a catastrophic harm is a rare
event with very serious consequences. These types of harms are also within the purview of
regulatory control, “as contingency emergency planning is critical to mitigating or even preventing
such harms” (Archibald et al., 2010, pp. 1-34.3). A poor outcome, where a risk assessment was not
completed in terms of the potential severity of consequence, could not only affect human health and
the environment (or other issue areas), but also public perception and public acceptance of
RBR/RBRD overall.

10
Alyssa Daku, CFR Regulatory Speaker Series, March 1, 2018
11
The Scotland & Northern Ireland Forum for Environmental Research (SNIFFER) published guidance on how to
assess the effectiveness of regulatory activities at low-risk sites, with a proposed good practice framework

24
Inherent low risk - The activity is not capable of producing intolerable harms/impacts and
stable operations are not likely to change in the periods between regulators’
strategy reviews.
Net low risk - stable The activity is capable of producing intolerable harms/impacts in the
periods between regulators’ strategy reviews but risks are reduced by
good management.
Inherent low risk but The activity is not capable (as presently organized) of producing
may change or intolerable harms/impacts in the periods between regulators’ strategy
accumulate reviews but operations (e.g. chemicals used) may change or there may
be numbers of such risks being created that create a cumulative problem
(e.g. because environmental absorption capacities are exceeded).
Net low risk but may The activity is capable of producing intolerable harms/impacts in the
change or accumulate periods between regulators’ strategy reviews but, at present, risks are
reduced by good management. That good management may, however,
change or there may be numbers of such risks being created that create
a cumulative problem.

Table 3: Types of low risk (excerpt from Black and Baldwin (2012, pp. 7-8))

25
 Section 3: Considerations for risk management

Risk management is the application and effectiveness of a variety of tools and strategies to achieve
the regulator’s objectives. These can be categorized within the REACT (regulatory, economic,
advisory, community-based, and technological) taxonomy of risk management actions, also
recognizing interrelationships between these five domains (i.e., a regulatory approach may require
action in one or more of the other four categories) (Krewski et al., 2007, Krewski et al., 2014).

RBRD interventions are proportionate to the identified likelihood and severity of the
consequence(s) of the risks; they are tailored to both the nature of the risk and the behavioural
drivers, including the attitude of the regulated entity and their capacity to comply based on the
business activities, size, and type (Australian Government, 2011, Black and Baldwin, 2012, NSW
Government, 2016, Office of the National Rail Safety Regulator, 2016). Regulators may use a
combination of strategies to engage in cooperation and responsiveness, and in so doing to
encourage regulated entities to go beyond compliance to help meet regulator’s objectives
(Gunningham, 2015). With a focus on compliance and enforcement as one strategy, this section
considers the continuum of these approaches, their prioritization, and monitoring and verification
within the risk management phase. However, if evidence suggests that a tool or approach is
ineffective, the evaluation, review and adjustment process (step 10, Table 2, discussed in section 5)
should reflect on the regulation itself, possibly instigating an oversight process to review the
regulation and result in an alternative approach or new approach if a RBR is initiated from the start.

a. Compliance tools and enforcement mechanisms

In the frameworks reviewed here, risk management is depicted in a continuum of compliance and
enforcement mechanisms, including some of the domains of the REACT framework. Compliance
and enforcement needs to be informed and validated, data driven and evidence-based, reliable,
defendable, and repeatable12. Nevertheless, while risk assessment activities, including scoring
regulated entity’s profile and activities, will detect high-risk actors, Gunningham (2015) suggested
RBRD does not identify specific risk management options that necessarily secure compliance.

The Alberta Energy Regulator’s Integrated Compliance Assurance Framework emphasizes shared
stewardship, continuous improvement, and innovative approaches and tools (Alberta Energy
Regulator, 2016b). For them, the model for an effective program identifies activities within the
three pillars of education, prevention, and enforcement (Figure 9). An operating principle of the
AER is fairness:
“Compliance activities will afford all parties a fair process and compliance decisions will be
made by objective decision makers. Noncompliance with regulatory requirements will be
addressed in a timely manner according to the statutory authority within legislation,
processes, and procedures” (Alberta Energy Regulator, 2016b, p. 5).

The following list of “soft” (to achieve voluntary compliance) and “hard/strong (regulatory
enforcement) risk management options has been gleaned from several sources (Alberta Energy
Regulator, 2016b, Australian Government, 2011, Environment Protection Authority Victoria, 2017,
European Aviation Safety Agency, 2016, Office of the National Rail Safety Regulator, 2016). Of note,
each jurisdiction’s legislative framework may or may not permit this range of activities.

“Soft” risk management options to achieve compliance

12
Fred Gaspar, Canadian Transportation Board, February 7, 2018 CFR workshop

26
a) Educational campaigns and providing advice are often described as key activities to promote and
achieve voluntary compliance. Activities could occur at the operator or industry (sector) scale and
are not legally binding. Options include in-person presentations, workshops, and operator-
awareness sessions; publications such as fact sheets, brochures, website pages, news releases, and

articles.

Figure 9: Alberta Energy Regulator compliance assurance model (2016b)

Education and providing advice are appropriate where there is a lack of awareness or
misinterpretation; degree of harm or potential harm is minimal (low risk); breaches may be
administrative with no material impact; the operator demonstrates a willingness to take action; and
improvements will ensure compliance.

The recently published OECD (2018) discussion paper “Enforcement and Inspections Toolkit” also
considers the practice of “assured advice”. Legal guarantees could be given to regulated entities
such that if official regulator advice is followed, the regulated entity “will not be held in breach of
their duties, even if at a later point another official reaches a different conclusion” (p. 31).

Promotion can be a stand-alone activity. Where a new regulatory requirement is approved,

promotion with regulated entities is used to make them aware of the requirement; understand the
requirement and consequences of noncompliance; and be put in a position to be able to comply.

b) Reporting might include performance report cards and industry advisories (compliance,
incident, and enforcement reporting); or based on other guidance such as industry best-
management practices. Non-conformance reporting may be appropriate if there is no immediate

27
threat to safety or the non-conformance is considered relatively minor; and the operator has
demonstrated a willingness and capability to address the breach within its safety management
system procedures. The expectation is that the operator will record and address any non-
conformances through their own audit/review/corrective action processes, as part of its safety
management system. Further non-conformance could lead to enforcement.

c) Consumer information - naming and shaming

d) Standard setting may be encouraged for the value chain of sector-based activities (see also
section 2 and 4)

e) Financial incentives could be made available to regulated entities with consistent good
compliance record, as a counter-balance of targeted enforcement for entities with consistent non-
compliance

f) Demonstrated intervention, such as increased staff training or involving more staff dedicated to a
safety management system

g) Collaborative problem solving/conversations

h) Industry research and development

“Hard/Strong” risk management enforcement options

a) Inspections and investigations - may target specific sectors or sub-sectors (possibly based on
monitoring activities). Suggested frequency may be pre-determined. As an example, the US Food
Safety Modernization Act of 2010 required that high-risk facilities be inspected within the first five
years of enactment and once every three years following. The non-high risk facilities were to be
subject to inspection once in the first seven years; then every five years (Drew and Clydesdale,
2015)

b) Enforcement penalties can be effective in ensuring that no party benefits from non-compliance,
deterring other potential noncompliant parties, and increasing an awareness of the importance of
environmental protection, public safety, or orderly development (Alberta Energy Regulator, 2016b)

c) Administrative responses (e.g., warnings, administrative sanctions, orders, directions)

d) Administrative Monetary Penalties (AMP) (not force of law; may include right to appeal) - of
note, if a fine is too high, AMPs may be contested. Questions in implementation include how to
apply discretion while being consistent

e) Certificate (license or permit) revocation

f) Enforceable voluntary undertakings (proposed by operator; if regulator agrees, then becomes

legally binding agreement)

g) Responsive sanctioning uses punishment to improve environmental outcomes (Environment

Protection Authority Victoria, 2017, p. 21). It is particularly relevant to the use of enforceable
undertakings and court-imposed alternative penalty orders. Alternative penalty orders allow the
court to require an offender to carry out a specified project to restore or enhance the environment,
rather than issue a fine or warning.

28
g) Prosecution, including charges, potential fines and/or other penalties

Prioritizing the approach

The mission of regulators is seen as the maximization of compliance levels, and not the systematic
detection and punishment of each and every violation. Following the risk control options analysis
(section 2), criteria can be used to determine the chosen risk management strategy from the soft
and hard options presented here. OECD (2018, p. 17) suggests that “enforcement requires
differentiation based on the track record of the operator, on the risk assessment (damages that the
violation has already caused and/or is likely to cause, considering also the broader compliance
context in the establishment), and on the potential effectiveness of different options”.

The frameworks emphasize progressive actions with a staged approach such that hard options
occur after a progression through soft options in order to improve understanding of the regulated
entity first, as well as to develop trust. The NSW Government (2016) framework indicates
escalating response (Figure 10) (but only for what are included here as hard regulatory options,
even for unintentional non-compliance).

Figure 10: Matching regulatory tools with attitudes and behaviour (NSW Government, 2016)

Regulators can establish thresholds where an occurrence of non-compliance moves the response
from one type of enforcement to another. The State of Victoria’s Environmental Protection
Authority uses a heat map matrix to illustrate increasing (regulatory) enforcement sanctions
depending on the culpability of the offender (Figure 11). The guidance includes a lengthy
description of what each of these enforcement responses might entail. Moreover, Archibald et al.

29
(2010, pp. 1-48) argue, relating again to instances of LPHI discussed in section 2, that “at the
highest level of potential harm to human health and or safety, a threshold is passed, such that the
weighing of the probability of non-compliance is not appropriate.

Figure 11: State of Victoria, Australia, Environment Protection Authority regulatory enforcement
response (Environment Protection Authority Victoria, 2017)

Gunningham (2015), Australian Government (2011), and the Office of the National Rail Safety
Regulator (2016) provide considerations for compliance and enforcement action(s):
 operator’s history of compliance and incidents
 the compliance tools at disposal
 use of approved codes of practice - providing operators with practical advice on compliance
and having a special legal status
 like situations treated in like manner; circumstances of each situation will be taken into
account and considered on own merits
 effectiveness of each tool in reducing social or economic harm
 speed of resolution
 efficiency, doing so at least cost
 comparable situations will have comparable outcomes
 legitimacy or political acceptability, including positive public perceptions of the regulator

The European Aviation Safety Agency (2016) might limit “hard/strong” options to circumstances of
gross negligence, demonstrated deliberate unwillingness to act, a significant decrease in overall
safety performance of the organization, or a continuously inadequate safety management system.
The Alberta Energy Regulator (2016b) considers the impact of compliance options on the regulated
entity, whereby requirements are set to balance the social and economic costs. Similarly, NSW
Government (2016) suggests that the cost and time impacts on business also be considered in
assessing and implementing the most appropriate enforcement response.

30
A word about causation. The OECD (2018, p. 31) discusses causation, suggesting that improving
compliance requires the analysis of what hinders it. Several issues have been identified (Borley and
Page, 2016, OECD, 2018, Smith et al., 2016): lack of information and understanding, insufficient
(financial) resources, poor regulatory design, paperwork, lack of time, language, distrust of
legislation and enforcers, and lack of motivation. Borley and Page (2016) also found that regulated
entities welcomed the support gained by investigative visits, an issue that may be addressed with
other compliance tools listed in section 4. A caution was also raised, although based on a small
sample of small and medium sized enterprise, that while “health and safety is important … other
business pressures and legal requirements push it further down the hierarchy. If health and safety
were not a legal requirement or if inspections were stopped/reduced health and safety would
become less important to them” (Borley and Page, 2016, p. 156).

Another issue in prioritizing the approach is for the regulator to display due diligence. A fulsome
discussion of this in relation to legal requirements is beyond the scope of this report. As a start,
readers are referred to a chapter on due diligence within risk management decision-making in
Archibald et al. (2010).

In summary, the goal of compliance and enforcement, in consideration of the risk profile of the
organization and its leadership, is to work within psychological thresholds that deter non-
compliance; or, put in positive language, to determine the effectiveness of a compliance option such
that it causes behaviour to change. In the long term, NSW Government (2016, p. 48) suggest that
such “consistency and transparency in applying enforcement responses will assist to underpin
confidence in the regulator”. An outstanding question for regulators is how to apply contextual
discretion while exhibiting a consistent approach. Moreover, mitigated risks are still risks. “These
risks have been reduced but not eliminated. In practice, the controls put in place may fail from time
to time” (Government of Canada, 2017b).

b. Measurement, monitoring and verification

In addition to the use of indicators and metrics in risk assessment, a critical issue for risk
management is how to measure and monitor the compliance approach in terms of general
deterrence and effectiveness in risk mitigation.

A measurement dilemma has been flagged for RBRD reporting: while it is relatively easy to quantify
AMPs, inspections, and open investigations, additional flexibility is inherent in the contextual
approach to deal with non-compliant actors. Perhaps the regulator’s immediate objective is to
build a partnership or attempt to use a new tool rather than a measurable enforcement option
which is not quantifiable. OECD (2018) suggests that performance should be measured in terms of
trends in achieving social well-being (safety, health, environmental protection etc.) and, as a proxy
for these goals, against improvements in compliance. This question applies to many health and
environmental protection domains - how to measure the positive impact of a harm that does not
occur.

With respect to monitoring, NSW Government (2016, p. 55) suggested that “monitoring, reporting
and continual improvement are the mechanics through which the benefits of the framework are
capitalised and maximised over time”. Continuous monitoring is required in order to feed into the
evaluation, review and adjustment of compliance programs (section 5). Monitoring will identify
whether there is a requirement to influence the law (new risk-based regulations) or industry
guidance (Office of the National Rail Safety Regulator, 2016).

31
Monitoring also supports RBRD by identifying activities within sectors, or specific regulated entities
at risk of unintentional or deliberate non-compliance and by providing information regulators use
to enhance their strategies aimed at preventing non-compliance (Australian Government, 2011) or
indeed, their strategies with respect to preventing harms.

Australia’s TGA framework (Australian Government, 2011) described legislated requirement of

both the regulated entity and regulator: for the former, “to monitor the performance of products in
the marketplace and with higher risk or serious health issues, to report problems to AU TGA in
timely manner”; and for the latter, to “monitor the market for signals of potential non-compliance
and then to determine the significance of signals and appropriate response”. Similarly, the BC Oil
and Gas Commission (2017) outlines monitoring to be completed by the permit holder in the case
of change management: that the permit holder develop and implement a systematic process for
identifying, evaluating, controlling and documenting a change to a facility, the specifications,
operations, standards, organization, activities, and legal requirements.

Frameworks suggest regulators’ monitoring activities could range from a phone call through to
formal written communications under sections of the legislative framework. Sources of
information in monitoring, detecting and investigating compliance could include:
 General surveillance
 Scheduled, random, and targeted inspections
o desk-top reviews
o safety related information
o sample testing
o full site visits - safety and efficacy
 Complaints
 Community and public reports
 Compliance audits - Regulated entity’s [safety] management system, standards and
procedures, and risk management
 Compliance investigations - Responding to problem reports, intelligence and tip-offs, and
adverse events

Moreover, regulators have established risk-based criteria in selecting which complaints or reports
to investigate (Office of the National Rail Safety Regulator, 2016). These mirror the risk assessment
phase, including:
 the severity of potential or actual harm
 knowledge of particular performance
 the regulator’s enforcement priorities (linked with risk tolerance)
 the likelihood of a successful enforcement action and meaningful improvement in
behaviour (seems questionable), and
 the contextual relevance of the event, including public concern.

A monitoring and verification plan can include wide ranging considerations (BC Oil and Gas
Commission, 2017): an assessment of the effectiveness of inspection method and technology,
previous integrity reviews, incident history, insufficient documentation, evaluation of anomalies,
time dependent considerations, current state of facility/equipment, and industry data. Decisions
are made for each regulated entity based on parameters and at an appropriate frequency to identify
any trend and to review the oversight programme, its cycle, and the safety objectives. For example,
guidance published by Government of Canada (2017b) requires that for high risk clients and

32
business relationships, more frequent monitoring is required, as well as enhanced measures in
business transactions.

Finally, where a verification process identifies non-compliance, a further assessment will help to
determine the appropriate compliance and enforcement response discussed above.
Responses suggested by the Alberta Energy Regulator (2016b) include:
 continued monitoring of the noncompliant party (verification)
 conducting activities to encourage compliance (promotion), and
 compelling compliance or deterring future noncompliance (enforcement)

33
 Section 4: Considerations for risk communication

The process of risk assessment and management has evolved and continues to evolve, from a
technocratic undertaking between government and industry representatives to now including, as
best practice, risk communication, engagement, and transparency with a broad stakeholder
audience. Paoli and Wiles (2015) suggested risk communication is necessary to support
participatory decision approaches, help determine tolerable risk levels, prepare risk information
for decision makers, and provide accurate and authoritative information in the event of an
emergency.

While Arnstein’s Ladder of Citizen Participation may seem dated (Arnstein, 1969), the challenges for
regulators and regulated entities in reaching higher levels of engagement remain salient: from
“informing” and “consultation” through potential “partnership” and “citizen control”. Regulators’
risk issues include not meeting objectives of the legislative mandate, avoidable occurrences of
harm, and losing public and industry trust. For the regulated entity, Gunningham (2015) suggested
the multi-faceted “license to operate” is monitored by stakeholders as a social license (where
stakeholders may shame, resulting in adverse publicity); as an economic license (where
stakeholders could generate consumer boycotts); and as a regulatory license (through citizen
lawsuits).

The importance of meaningful stakeholder engagement in an open and transparent process cannot
be overstated both during RBRD implementation and regulatory review and adjustment (section 5).
Stakeholder engagement and transparency are the least developed considerations for RBRD in the
frameworks reviewed here (Table 1, section 1).

a. Stakeholder engagement
Risk assessment and management is based on a shared responsibility, with stakeholder
engagement occurring within RBR, RBRD implementation step, and an oversight process to review
the regulation. Coglianese (2015), in the comprehensive reporting on regulatory excellence,
includes external engagement as one of four key facets of a “regulator’s organization and actions
that affect its performance” (p. v):
“External engagement. All things being equal, greater and earlier opportunities for public
engagement are better. Such engagement should be empathic. Listening is essential.”

In this instance, the word ‘stakeholder’ implies external organizations and the general public.
“The management of risks [to public health] is a value-driven exercise that must be informed by
and must respond to the views of the public, just as it must call on the best that science can offer
(The Honourable D.R. O’Conner, Report of the Walkerton Inquiry: The Events of May 2000 and
Related Issues, cited in Archibald et al. (2010, pp. 1-35)).

Gunningham (2015), in providing seven signposts of excellent regulators, suggested the need for
different strategies to engage effectively under different circumstances. Australia’s TGA provides a
list of engagement options for different sets of stakeholders: internal, external, community,
healthcare practitioners, industry, state and territorial governments, and international regulatory
bodies. The suggestion (2011, p. 43) is that:
“Communication and consultation within the TGA and with external stakeholders ensures
that there is a continuous feedback loop and that information from a very wide variety of

34
 sources is available to the TGA to assist it in its role in risk assessment and management of
medicines”.

Most frameworks reviewed here include reference to stakeholder engagement but not necessarily
in an elaborated form. Questions posed by Coglianese (2015) ask:
 Does the regulator generally provide opportunities for participation by any member of the
public that is concerned with or will be affected by its decisions?
 Does the regulator reach out to and welcome input by all individuals, organizations, and
communities that are interested in or affected by its decisions?

With respect to regulated entities, the Enforcement Inspections Toolkit discussion paper (OECD,
2018) suggests that key strategic decisions can benefit from stakeholder engagement, either
through formal ad hoc consultations or with an external advisory board-type structure. The
regulator needs to clearly communicate overall objectives and expectations in order to encourage
understanding and accountability (BC Oil and Gas Commission, 2017, Government of Canada,
2017b, Nicholls, 2015). Information, advice and guidance can be provided through practical and
easy-to-find guidance documents as well as through active outreach on-the-ground (OECD, 2018).
Moreover, “requirements that are set in consultation with stakeholders generally have higher levels
of voluntary compliance”(Alberta Energy Regulator, 2016b, p. 7).

Bombardier13 noted that at the federal level, engagement includes working with others and
investment in partnerships -- attention not just when the regulator needs the working relationship
but also to maintain relationships and networks. A caution here, however, is that the regulator
cannot be seen as “working with industry”. In addition to the outstanding issue discussed in section
3, about how to apply contextual discretion while exhibiting a consistent compliance and
enforcement approach, a balance in engagement between the regulator and regulated entity needs
to be struck in order to maintain public confidence.

One further consideration for engagement is the benefit in workplace safety for a regulated entity
engaged in internal stakeholder discussions, perhaps especially for higher risk occupational
environments. As found by Poplin et al. (2008), the employee (miner, in the case) can have greater
involvement and influence in the process of identifying and characterizing risk in a risk-based
operational environment. The increased awareness can help protect against personal risk issues
and may influence behaviour change.

Third parties
Stakeholder engagement includes consideration of the roles of third parties in risk assessment, risk
management (section 3), and broad elements in support of effective implementation (section 5).
Gunningham (2015) suggested that “an excellent regulator therefore facilitates, catalyzes, and
commandeers the participation of 2nd and 3rd parties to the cause of improving regulatory
outcomes”.

A limited review of third party participation finds the following types of involvement (and more
may be identified in subsequent research):
 by undertaking regulated activities on behalf of the regulated entity. While this may be a
business model decision of the regulated entity, the relationship may need to be taken into
account

13
CFR Compliance and Enforcement Community of Practice workshop, February 7, 2018

35
  as a conformity assessment body, such as a laboratory, inspection or certification body, that
provides data and/or assurances. The recommendation of the European Aviation Safety
Agency (2016) is that third party (compliance) assurance requires close supervision, and
the effectiveness of chosen risk control, at least at the outset, may need to be assessed
through additional internal or independent review
 in providing expert advice to the regulator or regulated entity. If the risk profile relies on
expert judgment, the European Aviation Safety Agency (2016) suggested that the decision
making should be made by consensus by a team of experts. For regulated entities,
providing education and advice to experts themselves will also help ensure the regulator’s
goals
 in setting industrial and operational standards for a sector. Gunningham (2015) suggested
that third parties should be harnessed as surrogate regulators. This is because, in setting
national or international standards, some homogeneity within trading partners and the
supply chain may be encouraged through peer pressure. Self-regulation through industry
association can reflect positively on management systems and garner respect for civil
society in a myriad of forms (usually NGOs and local community groups). The regulator has
a basis upon which to compare conformance or compliance. Indeed, some frameworks
reviewed here include adhering to third party standards within their guidance documents14
 at lower tier jurisdictions, as a primary authority15 - a legal partnership between businesses
and individual local authorities. Businesses receive assured and tailored advice where
operating at multiple premises
 as ombudspersons - a confidant, mediator or resource, to assist stakeholders in resolving or
addressing an issue prior to initiating a more formalized (complaint) process
 delegated authority - the assignment of responsibility to another regulator
 as a delegated assurance provider (e.g., insurer)

As stakeholders, engagement mechanisms can be developed for each of these third parties.

b. Open and transparent processes and outcomes

In returning to Coglianese (2015) with respect to regulatory excellence in external engagement at a
broad level, the earlier quote had more to say (as one of four key facets of a “regulator’s
organization and actions that affect its performance” (p. v)):
External engagement. All things being equal, greater and earlier opportunities for public
engagement are better. Such engagement should be empathic. Listening is essential. But so
too is reason-giving. Especially if a regulator must make a decision that will be opposed
or disfavored by some, the public deserve a full and forthright account of the policy
reasons underlying the decision.

Transparency (defined in terms of ease of access and the fullest possible disclosure) concerns the
risk assessment and management process as well as the regulators’ compliance and enforcement

14
The assurance protocol for inspection, monitoring and maintenance under the BCOGC Compliance
Assurance Protocol points to the American Petroleum Institute (API) Standard 570, which includes
provisions for determining inspection requirements based on risk, and API Recommended Practice 580 that
encourages the use of risk-based techniques to define inspection and testing requirements. The BCOGC also
follows other standards and recommended practice for their Fitness for Service Assessment.
15https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/664316/primary-

authority-overview.pdf

36
decisions themselves. A stated principle of the Alberta Energy Regulator (2016b, p. 5) compliance
assurance programs, procedures and activities is transparency: “Compliance information will be
open, transparent, and readily available to AER stakeholders. AER compliance activities will be
reported according to the provisions within the Responsible Energy Development Act”. The purpose
of public compliance reporting (p. 11) is:
 Improvement in “compliance rates by incentivizing compliant regulated parties, and by
deterring noncompliant behaviour.
 Meeting the AER’s statutory obligations and strategic goals for accountability and
transparency”.

There is therefore an implicit focus on documentation outlining a clear and fair process, with such
documentation being developed and completed with care. Three types of documents come to mind:
1) Policies and procedures for the risk assessment and management process. Paoli and Wiles
(2015) suggested a regulator might develop guidelines for risk communication activities
between regulated entities and their own clients or customers, as a condition of permit or
licence approval and compliance.
2) Clarity in the rules for compliance and enforcement decision-making, rights and obligations of
officials and of businesses (OECD, 2018). For example, thresholds for compliance and
enforcement could be developed and published in policy documents such that staff and
regulated entities are aware of the options and there is consistency in application. Moreover,
regulatory enforcement should be free of political interference.
3) Publicly available rationale for the decision made, including the ways in which the regulator
deals with uncertainty. As an example, “if a decision is reached to treat a given issue with more
(or less) intensity than its risk level estimated based on scientific evidence would warrant,
there should be full transparency and clarity on what values this decision was based on, and
what trade-offs were accepted as part of this decision” (OECD, 2018, p. 6).

In essence, the regulator needs to document and make transparent the mechanisms used in
problem solving in order to provide a basis for the risk-proportional decisions, including adequate
but bounded discretion. The Office of the National Rail Safety Regulator (2016, p. 12) and OECD
(2018) suggest, with respect to enforcement actions:
“To ensure consistency, transparency, accountability and impartiality in decision making, all
decisions and the reasoning for each decision shall be clearly documented. Each decision to
take enforcement action is to be supported by evidence that is sufficient to substantiate the
actions taken”.

Moreover, UK Better Regulation Delivery Office (2014) Regulators’ Code asks for businesses,
regulated bodies and citizens to challenge regulators who may not be acting in accordance with
published policies and standards. This office also monitors departmental activities in order to help
ensure proper implementation. A tension therefore appears to exist for the regulator in publishing
criteria to be used in risk-based decision making when RBRD also subscribes to flexibility in
compliance and enforcement options. This issue is not resolved here.

37
 Section 5: Other support for risk-based regulation and risk-based
regulatory delivery

Coglianese (2015) described a three-part ‘RegX’ approach for regulatory excellence: empathic
engagement in outward relationships, stellar competence in overall performance, and utmost
integrity in inner virtues - in short, a regulator that “listens, learns, and leads”. As the Community
of Practice in risk-based regulatory delivery evolves, there is an opportunity for members to help
each other grow in these ways.

Bombardier16 discussed best practices. Her view was informed in part by the work of Sparrow
(2008) - to understand what the regulator is trying to fix and why; to pick important problems and
fix them; to be fair in approach with a strong rationale; and to demonstrate innovation. This section
reflects on three issues that support effective approach overall: Leadership, Training, and Program
evaluation, review and adjustment.

a. Leadership
Demonstrated leadership by management executives is important - both for the regulators’ and
regulated entities’ organizations. Black (2008) provided the main lessons from earlier RBR
experiences. In starting out, the regulator should:
 Start with risks not rules
 Ensure the organisation has sufficient powers to implement the approach
 Beware of other regulatory or governmental policies which may contradict or hinder the
adoption of a risk based approach
 Ensure goals are known - it is worth doing, but don’t do it for the wrong reasons

More recently, Coglianese (2015) discussed excellence in traits, actions, and outcomes. Traits
concern culture and regulatory organization; actions concern selecting appropriate tools for the job
and using best practices in prioritizing risks and solving problems. These two facets then affect
outcomes (performance) in terms of internal management, external (and transparent) engagement,
priority-setting/decision-making, and problem solving.

With respect to integrating RBRD into corporate planning and management systems, the NSW
Government (2016) suggested first obtaining a baseline view of whether and/or the extent to
which the organization is using a risk-based regulatory approach, as well as an awareness of the
external business context. A diagnostic tools is available for all departments (NSW Government,
2018). Effective internal procedures will then help ensure a structured approach to planning and
gathering information, and measuring progress. Further, developing clarity with respect to
objectives will demonstrate how initiatives contribute over time, with improved transparency and
accountability.

Naturally, executive leadership does not work alone. RBR, RBRD, and the oversight (regulatory
review process) should be facilitated by a dedicated team of champions. An AER compliance
assurance principle reflects collaboration (Alberta Energy Regulator, 2016b, p. 5):

16
Speaker, The Regulatory Compliance and Enforcement Continuum, Community of Federal Regulators, February
7, 2018

38
 “Effective and efficient delivery of the AER mandate involves many interdependencies
across the organization. Staff will work together to ensure that the compliance assurance
system achieves the performance expectations and desired outcomes.”

In dealing with transition, Black (2008) suggested designing and implementing a risk based
framework will take time; organisational challenges are significant and should not be
underestimated; and to think beyond the risk assessment to how the regulator will respond.

Particularly with respect to the use of technological resources, regulators should, in the short term,
identify and improve areas that are within their control (for example, improving internal
processes). Then, in the longer term, regulators will identify gaps in capability and in implementing
solutions to alleviate these gaps (Black, 2008). As an additional comment, Gunningham (2015)
suggested positive attributes of self-evaluation in adaptive learning and resilience, and that this is
heavily dependent on the depth and accuracy of an agency’s statistical database and other
information sources (data issues also discussed in section 3).

b. Training
With respect to RBRD (implementation), competency is required for the regulator, the regulated
entity, and any third party personnel that may be involved. Although written with the regulated
entity in mind, a further suggestion by Government of Canada (2017b) applies to the regulator as
well: As risks will evolve over time, capacity and experience should also be expected to evolve.

Training is necessary at various levels of the organization. One model of competencies was
provided by the European Aviation Safety Agency (2016, p. 20) (Figure 12). Support and coaching
should be available during the initial phase of deployment and continuously over the long term.

Figure 12: Summary of competencies for risk-based oversight (European Aviation Safety Agency,
2016)

39
The New South Wales government guidance (NSW Government, 2016, p. 27) suggested the need to
“develop and embed mechanisms to underpin risk assessment quality, consistency and
transparency in applying judgement when completing risk assessments; embedding a consistent
understanding of the process throughout the agency; and underpinning confidence in the
regulator’s processes”. Members of the Society for Risk Analysis are writing Domain-Specific
Guidelines for Analyses Supporting Risk Management17.

Admittedly not exhaustive, some benefits of training include (Alberta Energy Regulator, 2016b,
European Aviation Safety Agency, 2016, NSW Government, 2016, OECD, 2014):
 General awareness and understanding
o objectives and strategies to achieve objectives
o the internal and external business context
o RBRD culture when interacting with industry
o role of the individual professionalism to ensure integrity, consistency and
transparency
 Risk assessment
o sharing methodologies and experiences
o use of expert judgment, especially when safety performance and “gut feeling” are
blended
 Risk management
o application of available compliance and enforcement tools

c. Evaluation, review and adjustment

As mentioned in the introduction, RBRD is not a static activity. Evaluation, review and adjustment
will assist with making continual improvements, all the while considering what is achievable
(Black, 2008). These activities are indicated as Step 10 in the generic risk assessment and
management process flowchart (Table 2, section 1), thus providing an opportunity to re-evaluate
policies and procedures. This is clearly an important global issue for effective implementation, in
identifying gaps and limitations and thereby linking back to other steps in the policy cycle (section
1). The continuum of compliance and enforcement does not always address risk issues adequately
and new or adjusted efforts become necessary. In this process, as with the sector, entity or activity
risk assessment and management process, open and transparent engagement with internal and
external stakeholders is highly recommended.

Evaluation, review and adjustment of the regulator’s processes and practices may affect (Alberta
Energy Regulator, 2016b, Australian Government, 2011, NSW Government, 2016):
 the regulator’s corporate and strategic direction
 policies and procedures
 resource allocations
 a re-examination of regulatory requirements
 understanding, adaptation and strengthening of evidence that underpins regulatory
initiatives; feedback mechanisms for data analysis and stakeholder engagement can assist
in testing RBRD assumptions
 work to identify and reprioritise risk issue areas over time - for example, high non-
compliant sectors or other complexities
 “whole of agency” structured approach to continuous improvement.

17
A project of the Applied Risk Management Specialty Group, Society for Risk Analysis

40
Similarly, the Alberta Energy Regulator (2016b) suggested lessons learned through evaluation and
review can help inform compliance decisions and enhance overall effectiveness and efficiency of
activities. A principle of the Alberta Energy Regulator (2016b, p. 5) is that the compliance
assurance framework is progressive:
“Knowledge gained from compliance assurance activities will be used to promote the
development of requirements that are clear, effective, and enforceable. Continuous
improvement, including incremental and transformational change, will be undertaken
systemically as an integral part of the Integrated Compliance Assurance Framework”.

Evaluation is also used to measure the results and implications of RBRD for regulated entities. The
most recent OECD (2018) discussion paper recommends jurisdictions develop an official policy on
evaluations in order to assess performance in meeting objectives; costs; and stakeholder
satisfaction, as examples. The Commonwealth of Australia has a legislated requirement for each
regulator to complete yearly performance assessments to assess whether the goals are being met
on a department by department basis (Australian Government, 2014). Questions during evaluation
could include (OECD, 2018):
 Is the performance of inspection and enforcement institutions (satisfaction, efficiency,
effectiveness) tracked regularly?
 Is the level of stakeholder (businesses, civil society) satisfaction and trust stable or
improving?
 Is the performance in terms of safeguarding social well-being and/or controlling risks
stable or improving (correcting for possible external shocks)?
 Is the efficiency (performance in terms of social well-being balanced with costs for the state
and burden for regulated entities) stable or improving?

41
 References

ALBERTA ENERGY REGULATOR (2016a). Compliance and Enforcement Program. Calgary,

AB: Alberta Energy Regulator. https://aer.ca/documents/manuals/Manual013.pdf
(Accessed 14 February 2018)
ALBERTA ENERGY REGULATOR (2016b). Integrated Compliance Assurance Framework.
Calgary, AB: Alberta Energy Regulator.
https://www.aer.ca/documents/enforcement/IntegratedComplianceAssuranceFramework_
February2016.pdf (Accessed 31 January 2018)
ARCHIBALD, T. L., JULL, K. E. & ROACH, K. W. (2010). Regulatory and Corporate
Liability: From Due Diligence to Risk Management, Student Edition, Aurora, ON:
Canada Law Book.
ARNSTEIN, S. R. (1969). A Ladder of Citizen Participation. Journal of the American Institute
of Planners, 35 (4), 216-224. 10.1080/01944366908977225
AUSTRALIAN GOVERNMENT (2011). The Therapeutic Goods Administration's risk
management approach to the regulation of therapeutic goods. HEALTH AND AGEING.
Canberra, AU: Australian Government. https://www.tga.gov.au/tgas-risk-management-
approach (Accessed 17 January 2018)
AUSTRALIAN GOVERNMENT (2013). Regulatory compliance framework. Therapeutic
Goods Administration. https://www.tga.gov.au/sites/default/files/compliance-
framework.pdf (Accessed 2 March 2018)
AUSTRALIAN GOVERNMENT (2014). Regulator Performance Framework. Commonwealth
of Australia.
https://www.cuttingredtape.gov.au/sites/default/files/documents/regulator_performance_f
ramework.pdf (Accessed 10 January 2018)
BALDWIN, R. & BLACK, J. (2016). Driving Priorities in Risk-based Regulation: What's the
Problem? Journal of Law and Society, 43 (4), 56-95.
BARLOW, S. M., BOOBIS, A. R., BRIDGES, J., COCKBURN, A., DEKANT, W.,
HEPBURN, P., HOUBEN, G. F., KÖNIG, J., NAUTA, M. J., SCHUERMANS, J. &
BÁNÁTI, D. (2015). The role of hazard- and risk-based approaches in ensuring food
safety. Trends in Food Science & Technology, 46 (2), 176-188.
10.1016/j.tifs.2015.10.007
BC OIL AND GAS COMMISSION (2017). Compliance Assurance Protocol Integrity
Management Program for Facilities Version 1.0. BCOGC.
https://www.bcogc.ca/node/13667/download (Accessed 17 January 2018)
BEAUSSIER, A. L., DEMERITT, D., GRIFFITHS, A. & ROTHSTEIN, H. (2016). Accounting
for failure: risk-based regulation and the problems of ensuring healthcare quality in the
NHS. Health Risk Soc, 18 (3-4), 205-224. 10.1080/13698575.2016.1192585
BLACK, J. (2008). Risk Based Regulation Presentation to OECD.
http://www.oecd.org/gov/regulatory-policy/44800375.pdf (Accessed 10 January 2018)
BLACK, J. & BALDWIN, R. (2012). When risk-based regulation aims low: A strategic
framework. Regulation & Governance, 6 (2), 131-148. 10.1111/j.1748-
5991.2012.01127.x
BORLEY, L. & PAGE, A. (2016). A reflection on the current local authority-led regulation
model: views from small- and medium-sized businesses. Policy and Practice in Health
and Safety, 14 (2), 144-162. 10.1080/14773996.2016.1255442

42
COGLIANESE, C. (2015). Listening Learning Leading: A framework for regulatory excellence.
Penn State: Alberta Energy Regulator and Penn Program on Regulation.
https://www.law.upenn.edu/live/files/4946-pprfinalconvenersreport.pdf (Accessed 10
January 2018)
DREW, C. A. & CLYDESDALE, F. M. (2015). New food safety law: effectiveness on the
ground. Crit Rev Food Sci Nutr, 55 (5), 689-700. 10.1080/10408398.2011.654368
ENVIRONMENT PROTECTION AUTHORITY VICTORIA (2017). Compliance and
Enforcement Policy. http://www.epa.vic.gov.au/~/media/Publications/1388%203.pdf
(Accessed 29 January 2018)
EUROPEAN AVIATION SAFETY AGENCY (2016). Practices for risk-based oversight. Koln,
GE. https://www.skybrary.aero/bookshelf/books/3699.pdf (Accessed 10 January 2018)
GOVERNMENT OF CANADA (2010a). Framework for the Management of Compliance.
TREASURY BOARD SECRETARIAT. Ottawa, ON. http://www.tbs-sct.gc.ca/pol/doc-
eng.aspx?id=17151 (Accessed 6 March 2018)
GOVERNMENT OF CANADA (2010b). Framework for the Management of Risk.
TREASURY BOARD SECRETARIAT. Ottawa, ON. http://www.tbs-sct.gc.ca/pol/doc-
eng.aspx?id=19422 (Accessed 6 March 2018)
GOVERNMENT OF CANADA (2017a). Compliance program requirements under the Proceeds
of Crime (Money Laundering) and Terrorist Financing Act (PCMLTFA) and associated
Regulations. FINANCIAL TRANSACTIONS AND REPORTS ANALYSIS CENTRE
OF CANADA. http://www.fintrac-canafe.gc.ca/guidance-directives/compliance-
conformite/Guide4/4-eng.asp (Accessed 31 January 2018)
GOVERNMENT OF CANADA (2017b). Guidance on the risk-based approach to combatting
money laundering and terrorist financing. FINANCIAL TRANSACTIONS AND
REPORTS ANALYSIS CENTRE OF CANADA. http://www.fintrac-
canafe.gc.ca/guidance-directives/compliance-conformite/rba/rba-eng.asp (Accessed 31
January 2017)
GOVERNMENT OF CANADA (2018). The Enforcement Response Panel III Federal
Regulatory Law Enforcement Symposium 2018. Ottawa, ON: Community of Federal
Regulators.
https://cfrevents.ca/uploads/files/FINAL%20Panel%203_FRLE%20Symposium%202018
%20Eng.pdf (Accessed 26 february 2018)
GUNNINGHAM, N. (2015). Compliance, Enforcement, and Regulatory Excellence. Australian
National University. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2929568
(Accessed 10 January 2018)
KOUTSOUMANIS, K. P. & ASPRIDOU, Z. (2016). Moving towards a risk-based food safety
management. Current Opinion in Food Science, 12, 36-41. 10.1016/j.cofs.2016.06.008
KREWSKI, D., HOGAN, V., TURNER, M. C., ZEMAN, P. L., MCDOWELL, I., EDWARDS,
N. & LOSOS, J. (2007). An integrated framework for risk management and population
health. Human and Ecological Risk Assessment: An International Journal, 13 (6), 1288-
1312. 10.1080/10807030701655798
KREWSKI, D., WESTPHAL, M., ANDERSEN, M. E., PAOLI, G. M., CHIU, W. A., AL-
ZOUGHOOL, M., CROTEAU, M. C., BURGOON, L. D. & COTE, I. (2014). A
framework for the next generation of risk science. Environ Health Perspect, 122 (8), 796-
805. 10.1289/ehp.1307260

43
NATIONAL RESEARCH COUNCIL (1983). Risk Assessment in the Federal Government:
Managing the process. Washington, DC: National Academy Press.
www.nap.edu/catalog/366.html (Accessed 16 August 2016)
NICHOLLS, A. (2015). The challenges and benefits of risk-based regulation in achieving
scheme outcomes. Injury Schemes Seminar. Adelaide, AU: Actuaries Institute.
https://www.actuaries.asn.au/Library/Events/ACS/2015/NichollsRegulation.pdf
(Accessed 10 January 2018)
NISULA, J. (2015). From Safety Indicators to Measuring Risk – the Risk-Guided Transport
Safety Agency. Risk in Motion S.A.S.
https://www.trafi.fi/filebank/a/1434456797/19018fa995da55930a03c3af8bc4f1ed/17872-
Nisula_From_Safety_Indicators_to_Measuring_Risk.pdf (Accessed 7 March 2018)
NSW GOVERNMENT (2016). Guidance for regulators to implement outcomes and risk-based
regulation. FINANCE SERVICES & INNOVATION. Sydney, AU: NSW.
https://www.finance.nsw.gov.au/sites/default/files/QRS_Outcomes_Risk_Based_Regulati
on_Guidelines.pdf (Accessed 10 January 2018)
NSW GOVERNMENT (2018). Regulation Diagnostic Tool.
http://www.treasury.sa.gov.au/__data/assets/excel_doc/0009/14949/NSW-Regulation-
Diagnostic-Tool_Sep-2014.xls (Accessed 20 January 2018)
OECD (2010). Risk and Regulatory Policy: Improving the governance of risk Paris, FR: OECD.
OECD (2014). Regulatory Enforcement and Inspections, OECD Best Practice Principles for
Regulatory Policy. OECD Publishing. http://www.oecd.org/gov/regulatory-
policy/enforcement-inspections.htm (Accessed 7 March 2018)
OECD (2018). OECD Enforcement and Inspections Toolkit. Draft for consultations.
http://www.oecd.org/gov/regulatory-policy/EI-Toolkit-Draft.docx (Accessed 8 March
2018)
OFFICE OF THE NATIONAL RAIL SAFETY REGULATOR (2016). Policy_Compliance and
Enforcement. Adelaide, SA: Office of the National Rail Safety Regulator.
https://www.onrsr.com.au/operators/compliance-and-enforcement (Accessed 29 January
2018)
PAOLI, G. & WILES, A. (2015). Key Analytical Capabilities of a Best-in-Class Regulator. Penn
Program on Regulation. https://www.law.upenn.edu/live/files/4710-paoliwiles-ppr-
researchpaper062015pdf (Accessed 22 January 2018)
PHAN, T. N. & BAIRD, K. (2015). The comprehensiveness of environmental management
systems: The influence of institutional pressures and the impact on environmental
performance. J Environ Manage, 160, 45-56. 10.1016/j.jenvman.2015.06.006
POPLIN, G. S., MILLER, H. B., RANGER-MOORE, J., BOFINGER, C. M., KURZIUS-
SPENCER, M., HARRIS, R. B. & BURGESS, J. L. (2008). International evaluation of
injury rates in coal mining: A comparison of risk and compliance-based regulatory
approaches. Safety Science, 46 (8), 1196-1204. 10.1016/j.ssci.2007.06.025
PUBLIC RISK MANAGEMENT (PRISM) INSTITUTE (2018 (unpublished)). International
Workshop on Risk Based Regulatory Delivery – Report on Workshop Summary and
Recommendations, December 13, 2017. (Accessed 15 January 2018)
SAFETY MANAGEMENT INTERNATIONAL COLLABORATION GROUP (2010).
Development of a Common Taxonomy for Hazards.
https://www.skybrary.aero/bookshelf/books/1779.pdf (Accessed 1 February 2018)

44
SAFETY MANAGEMENT INTERNATIONAL COLLABORATION GROUP (2013). Risk
Based Decision Making Principles. Safety Management International Collaboration
Group. https://skybrary.aero/bookshelf/books/2077.pdf (Accessed 25 January 2018)
SMITH, J., ROSS, K. & WHILEY, H. (2016). Australian Food Safety Policy Changes from a
"Command and Control" to an "Outcomes-Based" Approach: Reflection on the
Effectiveness of Its Implementation. Int J Environ Res Public Health, 13 (12)
10.3390/ijerph13121218
SOCIETY FOR RISK ANALYSIS (2015). SRA Glossary
http://www.sra.org/sites/default/files/pdf/SRA_glossary_20150622.pdf (Accessed 1
February 2018)
SPARROW, M. K. (2008). The Character of Harms, UK: CPI Group Ltd.
UK BETTER REGULATION DELIVERY OFFICE (2014). Regulators' Code. BUSINESS
INNOVATION & SKILLS.
https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/300126/14
-705-regulators-code.pdf (Accessed 7 March 2018)

45
 Appendices

(Intentionally left blank)

46
 Appendix 1 - Conceptual Frameworks

1. European Aviation Safety Agency’s practices for risk-based oversight (European Aviation Safety
Agency, 2016)

Based on a PLAN, DO, CHECK, ACT conceptual model in order to better visualise the feedback
loops, the following risk-based oversight scheme was proposed:

47
2. Australia’s risk management approach to the regulation of therapeutic goods (Australian
Government, 2011)

The TGA provides a generic risk management framework (AS/NZS ISO 31000:2009)

48
3. Australia’s Office of the National Rail Safety Regulator’s Policy - Compliance and Enforcement
(Office of the National Rail Safety Regulator, 2016)

Compliance and enforcement pyramid

49
4. New South Wales guidance for regulators to implement outcomes and risk-based regulation
(NSW Government, 2016)

Summary framework
5. Canada’s financial sector’s risk-based approach to combatting money laundering and terrorist

50
financing (Government of Canada, 2017b)

Guidance for financial sector entities that provide products and services through various delivery
channels.

51
6. Alberta Energy Regulator’s Integrated Compliance Assurance Framework (Alberta Energy
Regulator, 2016b)

AER’s vision and overall approach to assuring compliance

52
7. BC Oil and Gas Commission’s Compliance Assurance Protocol, Integrity Management Program
for Facilities (BC Oil and Gas Commission, 2017)

The compliance assurance process for IMPF aligns with the management systems approach and
applies to the entire life cycle of facilities.

53
BCOGC Integrity Management Program for Facilities Flowchart:
Requirements and Expectations related to Risk Management

54
55
 Appendix 2 - Glossary

This glossary of terminology and definitions was gleaned from a variety of sources:
Alberta Energy Regulator (2016b); Barlow et al. (2015); Government of Canada (2010b); OECD (2018); Paoli
and Wiles (2015); Safety Management International Collaboration Group (2010); and Society for Risk
Analysis (2015).

Acceptable/Unacceptable risk – the level of risk that individuals or groups are willing to accept/not
accept given the benefits gained. Judgement upon which a decision can be made to proceed or not
proceed with risk control options. Risk may be weighed against the intended benefit

Accountable - to ensure full accountability, the compliance of duty-holders, enforcement decisions

and the conduct of authorised officers will be explained and open to public scrutiny

Allocation (of resources) - decision to provide human and financial resources to initiatives in
proportion to the risk and complexity of regulated entities and behaviours

Attitude (to compliance) – a regulated entity could be committed to doing the right thing; trying to
do the right thing but don’t always succeed; don’t want to comply but will if made to; decision to be
non-compliant. Related to culture.

Comparative risk assessment (CRA) - see risk ranking and risk-based priority setting

Compliance – a program, process or procedure of a regulated entity to meet the regulatory

requirements such that risk or harm is prevented

Compliance promotion—any activity that increases awareness, that educates, motivates, or changes
behaviour, and that encourages voluntary compliance with a regulatory requirement

Consequence - actual or potential harm of a hazard that can be expressed qualitatively and/or
quantitatively. More than one consequence may evolve from an event or circumstance

Consistency - to take a similar approach in similar non-compliances and incidents to achieve similar
enforcement outcomes

Contribution analysis - test and validate how a regulator uses inputs (for example, resources) to
design initiatives, and how these are associated with changes in outputs and contribute to
outcomes (in short, intermediate and long-term)

Culture - an enduring set of values, norms, attitudes, and practices within an organization
concerned with minimizing exposure of the workforce and the general public to dangerous or
hazardous conditions

Enforcement - all activities of state structures (or structures delegated by the state) aimed at
promoting compliance (or deterring future noncompliance) and reaching regulations’ goals.
Activities may include: information, guidance and prevention; data collection and analysis;
inspections; enforcement actions in the narrower sense, i.e. warnings, administrative penalties or
sanctions, improvement notices (orders), fines, prosecutions etc.

56
Evidence-based - a risk assessment process should use established methods in collecting and
scrutinizing quantitative and qualitative evidence. Methods for combining evidence to reach
conclusions are particularly important

Exposure and exposure assessment - exposure and evaluation of exposure to a risk source/agent. A
recognized step in risk estimation

Fair and accountable - to act in best interest of the public and regulated entity

Gut feeling - see judgement

Harm - injury or damage. Harm can be to the environment or human health (physical or
psychological), technical processes or facilities, economic, organizational, political, etc.

Hazard - inherent property of an object or condition having the potential to cause harm (i.e. a risk
factor). Hazards can be associated with the environment or human health, technical processes,
materials, biota, information, the economy, an organization, politics

Hazard-based [regulatory] approach - to avoid a hazard with high potential for harm regardless of
how likely the outcomes might be

Hazard characterisation - qualitative and, wherever possible, quantitative description of the

inherent property of an agent or situation having the potential to cause adverse effects (harm).
Where possible, includes a dose-response assessment and attendant uncertainties. A step in risk
estimation

Hazard identification - a process to establish a list of hazards relevant to the activity, the causes and
characterisation. Will help create the hazard taxonomy. The first step in risk estimation

Hazard taxonomy - list of hazards pertinent to a sector or entity and used for data categorization
and analytical purposes

Heat map - see risk matrix

High risk - usually unacceptable level of risk. Activity cannot continue unless hazards are further
mitigated so that risk is reduced

Impact - see Consequence

Indicators - a data-based parameter used to monitor, assess and report on performance over time.
A good indicator is well-defined with clear links to regulatory outcomes, timely, and best available
at the most reasonable cost. Indicators can be used to establish clear baselines on which to monitor
change over time

Inherent risk - intrinsic risk of an event or circumstance that exists before the application of risk
controls or mitigation measures

Inspection – any type of visit or check conducted by authorised officials on products or business
premises, activities, documents etc.

57
Integrated risk management - a continuous, proactive and systematic process to understand,
manage and communicate risk from an organization-wide perspective. Supports strategic decision-
making that contributes to the achievement of an organization's overall objectives

Judgement - a risk assessment output is typically a subjective measure based on a combination of

quantitative and qualitative data, and the regulator’s own judgement, itself based on values, criteria
and trade-offs. Input from experts also include judgements

Likelihood - the frequency, in quantitative or qualitative terms, that a hazard may occur.
Synonymous with probability

Low risk - a level of risk in which the identified hazards are not usually required to be actively
managed, but are documented

Measures - see indicators

Non-compliance – a regulated entity’s program, process or procedure fails to meet regulatory

requirements

Open and transparent - fully document and clearly communicate to all interested persons
interested with the reasons for decisions

Organizational hazard - hazards which arise from an organization’s policies, priorities and the
manner in which work is carried out

Outcome - a result, usually valued or needed by the public, e.g., health, property, infrastructure,
cultural artifacts, fairness in commerce, human rights, animal welfare, functional banking systems

Outcomes-based approach - to assess both the efficiency and effectiveness of regulatory actions

Oversight - function performed by regulator to ensure regulated entity complies with standards,
regulations and procedures
Oversight cycle – frequency of assessment; should take into consideration the risk profile and
performance assessment

Permit holder – see regulated entity

Precautionary principle - where there are threats of serious or irreversible damage, lack of full
scientific certainty shall not be used as a reason for postponing cost-effective measures to prevent
environmental degradation

Priority setting - can be driven by quantitative analysis (e.g., maximizing incremental cost-
effectiveness); formal logic (scoring schemes); a deliberative process such as periodic review by
internal and external experts; or combination/hybrid approach. See also comparative risk analysis

Proactive methodology – evaluate risks after audits, inspections or reporting but before a harm has
taken place

Probability - see Likelihood

58
Proportionality - the scope, depth and rigour of regulatory measures and responses in proportion
to the apparent level of impact of the risk

Reactive methodology – analyzing hazards as a result of an incident or accident

Regulated entity – a person or business or organization subject to regulatory requirements. Same

as regulatee

Residual risk - level of risk that remains after the implementation of mitigation measures and
controls on inherent risk factors

Risk – the effect of uncertainty on objectives and outcomes. The concept integrates the likelihood of
an adverse event (of a hazard) and the severity of the consequence (harm), whether to health, the
environment, the related achievement of an organization’s objectives, or a breach of legislation.
The harm may come to individual entities or populations, usually relevant to the domain of
regulation. Often the risk is to vulnerable populations

Risk analysis - See risk assessment and risk management

Risk assessment - analytical and systematic process to identify, evaluate and estimate
[characterize] the level of risk for identified hazards, including exposure assessment and attendant
uncertainties. A risk assessment output is typically a subjective measure based on a combination of
quantitative and qualitative data, and the regulator’s own judgement. Results of risk assessment
inform risk management. Risk assessment can be applied to any situation. Synonymous with risk
analysis, a term used in some jurisdictions.

Risk-based oversight - regulators’ planning is driven by the combination of risk profile and safety
performance of the regulated entity. Similar to risk-based regulatory approach and performance-
based oversight

Risk-based regulation (regulatory delivery) - a competent authority ensures that applicable (often
health and safety-related) requirements are met by regulated entities based on a process to assess
risk of the business and its activities; risk mitigation through controls and measures are tailored to
the identified risks; ongoing monitoring in accordance with assessed level of risk. Compliance and
enforcement decisions are based on an understanding of the nature and severity of an event,
problem, or activity that has the potential to compromise strategic objectives and inform the
optimal response based on sound analysis that is transparent and defensible

Risk characterisation - qualitative and/or quantitative picture of the risk; i.e., a structured
statement of risk usually containing the elements: risk sources, causes, events, consequences,
uncertainty representations/measurements (for example probability distributions for different
categories of consequences – casualties, environmental damage, economic loss etc.) and the
knowledge that the judgements are based on

Risk communication - an exchange or sharing of risk-related data, information and knowledge

between and among interested parties, generally including regulators, experts, stakeholders,
consumers, media, and the general public. Risk communication should consider an understanding
of the perceptions, concerns and values that underlie the perspective

Risk control - activities that attempt to minimize risk. See also risk management

59
Risk evaluation - process of comparing the result of risk assessment (the risk) against (often)
benefit criteria to determine the significance and acceptability of the risk

Risk-informed approach - see also risk-based regulatory delivery - risk assessment and
management are built into existing governance and organizational structures, including business
planning, decision-making and operational processes

Risk matrix - a graphic display. Combines increasing likelihood and severity of consequence of
hazard on x and y axis; could also combine likelihood of compliance (based on risk profile of
organization) compared with risk to health and environment

Risk management - a systematic approach to set the best course of action under uncertainty by
identifying, assessing, understanding, making decisions on and communicating risk issues. Steps
include risk assessment, establishing risk controls in order to reduce the impact of risk to an
acceptable level, monitoring and evaluation. Decisions will be made on where to allocate resources
to identify non-compliance in higher and lower priority entities, and enforcement tool selection

Risk mitigation - risk reduction; process of risk control actions to reduce risk but not eliminate it.
In practice, risk controls may fail from time to time

Risk perception - a person’s subjective judgement or appraisal of risk

Risk policy - a plan for action of how to manage risk

Risk profile - elements of risk inherent to the nature and the operations of the regulated entity: the
specific nature of the organisation; the complexity of its activities; the risks stemming from the
activities carried out; results of past certification and/or oversight. When the risk profile relies on
expert judgement, decision making could include consensus by a team of experts. A risk profile can
also be completed for a sector of the economy to develop a deep understanding of the effects of
risks that sector participants must address

Risk register – list and description of key risks and effects arising from sector, entity or activities.
Could include risk description (cause/source, impacts, controls, stakeholders and
likelihood/consequence); primary risk owner; current rating; treatment description; treatment
owner; residual risk rating; risk review date

Risk response - refers to the continuum of measures of risk control that are developed and
implemented to address an identified risk

Risk tolerance - the willingness of an organization to accept or reject a given level of residual risk.
Clarity on risk tolerance at all levels of an organization is necessary to support risk-informed
decision-making and foster risk-informed approaches

Risk transfer – assign ownership of the hazard issue and mitigation approach to the organization or
operation most capable of managing it

Risk treatment – See Risk management

Safe - without unacceptable risk

60
Safety management system - a systematic approach to identify and manage hazards, including the
necessary organizational structures, accountability, responsibilities, policies and procedures such
that regulated entities manage their risks. Methods of management and control combine with or
offset inherent risk, whereby the safety management system can either exacerbate the inherent risk
or mitigate it

Safety performance - the demonstration of how effectively a regulated entity can mitigate its risks,
substantiated through the proven ability to comply with the applicable requirements; implement
and maintain effective safety management; identify and manage safety risks; achieve and maintain
safe operations. Safety performance is continuously changing over time, as it is the direct outcome
of the continued operations of the regulated entity

Stakeholder - organizations, groups of people or individuals who may be affected by a risk-related

decision and who could influence the risk-informed decision or implementation of risk mitigation
measures

Stewardship - the careful and responsible management of resources for the benefit of present and
future generations

Timeliness - the timeline for a risk assessment should be established in consultation with the
decision-makers to ensure that the risk assessment context is clear and realistic expectations are
maintained

Tolerable risk - risk that has not been reduced to the desired level however further reduction is
impracticable or the cost is disproportionate to the improvement that would be gained. Tolerable
risk may be reviewed over time, for example, when a new product or process is introduced or a
new hazard appears

Transparency - information on the objectives, evidence, process, conclusions and enforcement

during and following a risk assessment are made available to stakeholders and the interested public
(subject to valid information security constraints). Also applied to development of regulations and
standards. Lessons learned should be shared and promoted. Transparency builds credibility of,
and confidence in, the regulatory approach and processes

Uncertainty - the state, even partial, of a deficiency of information such that the true value of a
quantity, the future likelihood or consequence of an activity, is not known; imperfect or incomplete
information/knowledge. The type, source, degree and significance of uncertainty should be
described during a risk assessment and management process, with attention to potential impact on
the conclusions of the assessment and evaluation of any specific risk management option under
consideration

Voluntary compliance - the situation whereby regulated entities comply with regulatory
requirements of their own accord; while a notice of noncompliance may be issued, the regulator is
not compelling compliance through an enforcement action

Zero-risk - not seriously contemplated as an expected outcome as this does not exist

61
 Appendix 3 - Risk metrics, Risk ranking

Excerpts from Koutsoumanis and Aspridou (2016, p. 37) with a focus on food safety

Risk metrics
There are different ways of expressing risk in a risk assessment. Codex Alimentarius defines risk as
‘a function of the probability of an adverse health effect and the severity of that effect,
consequential to a hazard(s) in food’. The simplest metric that can be used to account for the
probability of an adverse effect in risk ranking is the number of adverse outcomes (e.g. illnesses,
hospitalizations, and deaths) associated with a single hazard in multiple foods. The number of
adverse outcomes can be estimated as ‘per serving’ or ‘per annum (and standardized for population
size (e.g. per 100 000 per year))’. The ‘per serving’ likelihood can be viewed as the risk that
individual consumers face when they eat a serving of a food. The ‘risk per annum’, on the other
hand, is a measure of the risk faced by a certain population (e.g. a country). The risk per annum is
greatly affected by the number of servings per year. In the case of multiple hazards, the challenge is
to find metrics to characterize the severity of the health outcomes associated with these hazards in
order to compare their overall health and/or economic impact. The DALY approach (Disability-
adjusted life year) was first developed by the World Health Organization’s Global Burden of Disease
(GBD) program to compare the risk of specific diseases in different countries. The DALY method
presumes perfect health for the entire life span and, therefore, measures the loss due to ill health.
Death, the worst possible health state, is assigned a disability weight of 1 while 0 represents the
best health state. To calculate the burden due to premature mortality, the number of life years lost
is compared to a standard life table. A number of approaches have been developed for the monetary
valuation of risk. In this case, the public health impact of foodborne disease is characterized by
health economics. The risk metrics can significantly affect the risk management decisions and thus,
their selection requires communication between the risk assessors and the risk managers.

Risk ranking
Policy makers and food safety authorities must deal with numerous food safety issues, often
simultaneously, and inevitably, resources are insufficient to manage all issues at any given time.
Setting priorities and allocating resources plays a crucial role in the decision-making process. A
‘priority’ issue is essentially one that is considered to be a matter of greater importance, and which
should thus, be addressed with more urgency and in precedence to other issues. Risk ranking in
food safety can be considered as a risk assessment exercise for ranking the combined probability of
food contamination, consumer exposure and public health impact of certain foodborne hazard–
matrix combinations. Two approaches can be adopted; the bottom up (forward) which is based on
exposure data and dose response relationship and the top-down (backward) approach which is
based on disease incidence and attribution data. Risk ranking has been recognized as the proper
starting point for risk-based priority setting and resources allocation, because it would permit
policy makers to focus attention on the most significant public health problems and develop
strategies for addressing them. The objective of the risk ranking in the general risk management
framework is the evaluation of the perceived relative level of risk that each issue presents to

62
consumers, so that the risk management resources can be optimally distributed to reduce overall
food-borne public health risks. Several (semi)-quantitative risk ranking tools are available,
including among others, FDA-iRISK [25], microHibro, Risk Ranger, and sQMRA. EFSA recently
developed a conceptual framework with nine separate stages leading to a structured, transparent
and consistent approach in risk ranking.

63
 Appendix 4 - Potential tools for low risk sites or activities

Excerpt from Black and Baldwin (2012, pp. 21-22)

Screening and rule-based strategies

1. Exemptions without notification or registration
2. Exemptions with notification or registration
3. Registration plus conditions/rules; permit and licensing systems
4. Application of general binding rules without notification/registration

Monitoring tools
5. Frequency adjusted inspections or monitoring
6. Regulatory audits
7. Themed inspections or monitoring
8. Random inspections or monitoring
9. Advice and assistance visits
10. Reactive investigations, responding to complaints, whistleblowing or post-incident
investigations
11. Surveillance
12. Benchmarking or ‘yard sticking’ strategies
13. Measuring indirect/proxy outcomes
14. Self-monitoring and self-certification by regulated firms
15. Management based strategies including mandatory performance disclosure by regulated firms
16. Third-party monitoring
17. Information and inspection sharing regimes

Engagement and incentive strategies

18. Information campaigns; generic advice and recommendations (including codes and guidance)
19. Dialogue with interested parties
20. Industry or NGO / interested party-led solutions
21. Multi-agency approaches
22. Incentive strategies

64