NDRs: Models and Purposes

National Data Repositories, what they are, their purpose

and general models for their delivery.

Discussion Document
(Restricted Distribution)

27th May 2010

Malcolm Fleming
Chief Executive
Common Data Access Limited

CONTENTS
1
2

INTRODUCTION ................................................................................................. 1
WHAT IS AN NDR? ............................................................................................ 2
2.1 General Definition ......................................................................................... 2
2.2

National ......................................................................................................... 2

2.3

Data .............................................................................................................. 3

2.4

Repository ..................................................................................................... 4

PURPOSE OF AN NDR ...................................................................................... 5

3.1 Regulations & Licence Conditions................................................................. 5
3.2

Promotion and Facilitation of Oil and Gas Development ............................... 5

3.3

Active Data Storage & Management ............................................................. 5

3.4

Preservation of the National Legacy ............................................................. 6

NDR MODELS..................................................................................................... 7
4.1 Generic Model ............................................................................................... 7
4.2

Funding Models............................................................................................. 7

4.3

Operational Models ....................................................................................... 8

4.4

Commercial Models ...................................................................................... 9

RECOMMENDATIONS & CONCLUSIONS ...................................................... 10

5.1 NDR Scope and Purpose ............................................................................ 10
5.2

Data Map NDR Components in Place ...................................................... 10

5.3

NDR Models ................................................................................................ 10

5.4

Requirements Specifications ....................................................................... 11

5.5

Solutions ..................................................................................................... 11

Appendices:
A
Principal National Data Repositories
B
NDR Summaries
C
NDR Performance Assessment

INTRODUCTION
The establishment of purpose-designed NDRs is a growing phenomenon (see
Appendix A). It is motivated by a countrys desire to promote investment in oil and
gas and by the need to collect and manage the petrotechnical information which
supports both this task and the regulatory regime in place for existing licensees.
There are several implementation models available for NDRs (see 4 and Appendix
B) and the majority place the state firmly at the centre. Some NDRs are deliberately
organised, others have evolved naturally over the years.
This paper starts by defining what is an NDR (2 and 3) and its scope and purpose
(3).
This report concludes that successful NDRs must have a clear purpose and vision. The
state regulator must take the lead in promoting a shared understanding of the
importance of efficiently managing licence data to maximize national oil and gas
potential.

Filename: NDR Models

Page 1

WHAT IS AN NDR?

2.1

General Definition
Although the majority of NDRs are centralised and are public there is no single
applicable model. They may for example be distributed and/or partly private. NDRs
may also be wholly digital, wholly physical or (as is more usual) a blend of both.
Appendix A lists the principal known operating NDRs. Information on the range of
underlying funding models and relationships with the systems vendors is given in 4.
Appendix B indicates the data scope of these NDRs and some other useful
comparative characteristics.
The NDR Work Group1 describes an NDR as a government and regulatory agency
which collects, stores and disseminates quality oil and natural gas related
[petrotechnical] data [] including seismic, drilling, production and reservoir data2.
This is a reasonable definition, however each of the three components of the NDR
abbreviation warrants a little closer examination.

2.2

National
The term National is defined in the OED as of, or pertaining to a nation or a
country, as a whole.
In the context of an NDR the geographical scope may be a subset of the nation
where there is separation caused by geographical, geological and/or political
boundaries. This is the case with the Western Australian NDR, in Russia, the United
States and in the UK3.

Figure 1: National

1
2
3

The National Data Repositories Work Group is part of the Energistics (www.energistics.org) eRegulatory Special Interest Group
(Energistics was formerly known as POSC).
http://www.energistics.org/posc/NewsBot.asp?MODE=VIEW&ID=1156&SnID=665912804, August 2009.
The UK Onshore and the UK Continental Shelf are regulated and operated separately.

Filename: NDR Models

Page 2

2.3

Data
The scope of data held within an NDR generally includes the following:
PETROTECHNICAL DATA
This term encompasses all geological, geophysical and reservoir data associated with
oil and gas exploration, appraisal, production and development. In the NDR context it
tends to refer to raw uninterpreted data but this is usually a grey area which must be
determined within the context of: (a) the policies and business objectives which
underpin the NDR (see 3); and (b) stakeholders capacity to fund the NDR over the
lifecycle of the term of exploitation of the nations oil and gas resources (and
beyond).
CULTURAL & HEADER INFORMATION
This group includes spatial and other descriptive attributes of licences, coastlines and
other natural boundaries and national and international political boundaries.
It also includes header descriptions for seismic surveys, infrastructure,
licences/permits and other types of business object used in industry and held in the
NDR. For example the well header is a collection of attributes that describe a well,
including spatial information (e.g. coordinates, significant depths, datums), temporal
information (e.g. date spudded, dated reached TD and date suspended) and
information about the stakeholders in the well (operator and perhaps partner interests).
METADATA
The term metadata is commonly defined as data about data. Metadata describes the
history of the data within the repository and includes attributes such as date added,
unique ID, ownership and entitlement rights, a catalogue description, amendment
dates and physical or logical system addresses.
Metadata can also extend to include prices and commercial transaction histories if
appropriate. Metadata management is essential for the effective management of a
repository of any kind.

Figure 2: 'Data'

Filename: NDR Models

Page 3

2.4

Repository
The dictionary definition of Repository is a place or container in which things can
be stored for safety. The term implies the singular but more usually will include a
series of related stores or independent collections.
The word Repository implies secure storage but this function is redundant without
systems and methods to support access to the information held. The word Centre is
probably a more accurate term however the use of the word Repository is now wellestablished in the NDR community.

Figure 3: 'Repository'

Filename: NDR Models

Page 4

PURPOSE OF AN NDR
The purpose of an NDR varies at the detailed level but generally reflects a set of
policies and business objectives which can be distilled down to the factors described
below. The specific requirements and performance of any NDR should be considered
within the context of these factors.

3.1

Regulations & Licence Conditions

Licence conditions set out the licensees and the regulators rights and obligations
with respect to petrotechnical data and it is this information which tends to fall within
the scope of an NDR. Licence conditions also spell out (whether implicitly or
explicitly) where ownership of data resides either with the State or with the
Licensee(s).
Regulations build upon licence conditions to specify what licence information is to be
reported (sent) to the regulator (or regulators) and how, when and in what form it
must be reported.
The key success factors in this area are:
Coherent, sensible and well-publicised regulations;
Practical data standards (and in particular standards for data exchange) which are
prescribed and enforced by the regulator;
Fair, effective and timely enforcement of the rules;
Clarity on data ownership and associated rights; and
Material consequences for non-compliance.
These success factors can be used as criteria against which the performance of an
NDR may be assessed (see Appendix C).

3.2

Promotion and Facilitation of Oil and Gas Development

The NDR generally has an important strategic function (in concert with other
measures employed by the state) in promoting licence rounds and attracting
international investment.
This is a competitive function and has the following key success factors:
Global reach to encourage real interest from as wide a field as possible;
Simple access to relevant, quality data, published at a fair price; and
Linkage to licence conditions and a Doing Business in guide.
As above, these success factors can be used to assess NDR performance (see
Appendix C).

3.3

Active Data Storage & Management

The NDR usually has a day-to-day function, serving government departments, the
regulatory authorities, NOCs and operating oil companies and service companies. In
its distributed form, the NDR may include decentralised facilities operated by
licensees themselves (or perhaps on a collaborative basis as in the UK).
The keys to success in this area are:
Secure storage of a minimum number of instances of the data;
Knowledge of all relevant information available;
Simple data access;
Data usability (formats, media and completeness);
Acceptable costs and terms of use;

Filename: NDR Models

Page 5

Process alignment with the states policies and arrangements for promotion (see
3.2 above);
Process alignment with the states policies and arrangements for long term
preservation (see 3.4); and
Regulatory compliance.

These criteria can be used to assess NDR performance (see Appendix C).
3.4

Preservation of the National Legacy

The NDR has a longer term role for the preservation and publication of (selected)
petrotechnical data. Here the keys to success are:
A practical channel for archiving all relevant data from the active collections
(see 3.3 above);
A pragmatic policy of data selection (it is usually neither economically feasible
nor scientifically necessary to keep everything);
Secure storage conditions that are appropriate to each of the data types stored
(cores and digital tapes require different conditions for example);
Assured evergreen preservation in a usable format on a usable medium; and
Sustainable, long term funding.
These factors can be used to assess the performance of an NDR (see Appendix C).

Filename: NDR Models

Page 6

NDR MODELS

4.1

Generic Model
The four purposes described individually in the section above can be linked together
within a single, integrated generic model (see Figure 4).

Figure 4: Generic NDR Model

If it is correct then it ought to be possible to map any NDR into the model (and this is
true for the UKCS distributed NDR).
4.2

Funding Models
The most common model (see also Appendix B) is for the state to fund the NDR and
this seems always to be the case where licence data is owned by the state. State
funding may come from a central pot or (as is the case in New Zealand for example)
is co-funded from permit application fees.
The scale of funding is difficult to ascertain but some examples are available (the
Netherlands TNO was funded by the state to the tune of 15 million in 2007 for
example).
In Norway, Diskos is funded by licensees through subscriptions and transaction fees
but those parts of the national collection that are outside Diskos are funded either by
licensees themselves (seismic field tapes) or by the state (geological samples) in a
way which closely maps to the UKCS model.
NDRs in developing countries often receive support through soft loans or funded
advice from US Aid, World Bank, NorAd and other agencies.
The UK Onshore Geophysical Library (UKOGL) is funded as a charity.

Filename: NDR Models

Page 7

Figure 5: NDR Funding Models

4.3

Operational Models
All NDRs are distributed to some extent but the split is usually along the lines of
digital versus physical data (geological samples being the prime example). The
majority however attempt to collocate all digital petrotechnical data. Due to the data
volumes involved, the smaller NDRs have more successfully integrated seismic field
data into a unified NDR scope.
Schlumberger and Landmark are the main technology and service providers (see
Appendix B). Most NDRs purchase their own hardware and software licences and
have entered into training and support arrangements with their vendor of choice. This
is often the preferred model for start-up NDRs. A managed service model has been
successfully adopted by CDA and is appropriate for a mature NDR.

Figure 6: NDR Operational Models

Filename: NDR Models

Page 8

4.4

Commercial Models
Most NDRs have entered into fixed contract terms with their vendors yet (it seems)
only CDA and Diskos have experience of retendering and the challenges of changing
service provider.
It is common practice for many NDRs to make data available free-of-charge and only
where there are costs to be recovered is it necessary for users to buy licence data from
NDRs.
Where the state owns the data then it can truly be made public after an agreed term
of confidentiality, completely avoiding any data licensing and rights issues.

Figure 7: NDR Commercial Models

Filename: NDR Models

Page 9

RECOMMENDATIONS & CONCLUSIONS

5.1

NDR Scope and Purpose

Determining the scope (see Section 2.3) and the purpose (see Section 3) of an NDR an
(as iterative exercise) must be the starting point for its creation. Failure to approach
an NDR in this way inevitably leads to a compromised or failed implementation.
This approach is a useful exercise for new and established NDRs alike.

Figure 8: Conclusions

5.2

Data Map NDR Components in Place

In the great majority of cases, the component parts of the general NDR model (see
Figure 4) are already in place but they have not been recognised as such. This is
certainly true of the UKCS.
It is appropriate therefore first to create an inventory, capturing the following at a very
high level:

Petrotechnical data class (e.g. well data)

Data type (e.g. well logs)
Data sub-type (e.g. raw digital well log curves)
Physical format /medium (e.g. physical, digital)
Repository details

This exercise will provide a data map of all known stores of petrotechnical data and
will reveal general gaps and duplications in the aggregated national collection.
5.3

NDR Models
Careful thought must be given to the funding model (see Section 4.2) for the NDR
and data ownership (whether by the state or by licensees) is one of the key influencers
in determining this.

Filename: NDR Models

Page 10

The preferred commercial model (see Section 4.4) and operational model (see
Section 4.3) must also be seriously considered at the early stages of the establishment
of an NDR and periodically thereafter.
5.4

Requirements Specifications
It perhaps goes without saying that the requirements of all NDR stakeholders must be
canvassed and understood before any attempt at implementation is made.

5.5

Solutions
Technological solutions for NDRs are now mature and available but vendor selection
must follow from user requirements and a clear understanding of the purpose of the
NDR. It is extremely unwise to select the technology first.

Malcolm Fleming
27th May 2010

Filename: NDR Models

Page 11

Principal National Data Repositories

Appendix A

Below is a list of the principal NDRs:

Angola ANDR (Sonangol) (www.sonangol.co.ao)

Australia (WA) WAPIMS (Department of Industry and Resources - DoIR)
(www.dmp.wa.gov.au/4187.aspx)
Brazil BDEP (ANP) (www.anp.gov.br)
Cameroon
Canada (Nova Scotia) Digital Data Management Center (CNSOPB)
(www.cnsopbdmc.ca)
Colombia EPIS Exploration & Production Information Services (ANH)
(www.epis.com.co).
Ghana
India NDR (DGH)
Iraq
Kazakhstan Petrodata Kazakhstan (JV between Halliburton & Kazakhstan Institute
of Oil & Gas) (http://www.petrodata.kz/page.php?lang=2)
Kenya NDC (National Oil Corporation of Kenya)
Malaysia E&P Data Repository (Petronas)
Mexico @ditep (Pemex)
Mozambique NAPD (National Petroleum Institute - INP)
Namibia - NAMCOR
New Zealand Crown Minerals
(http://www.crownminerals.govt.nz/cms/petroleum/technical-data)
Netherlands DINO (TNO)
(http://www.tno.nl/content.cfm?context=markten&content=product&laag1=188&laag2=2
53&item_id=723)
Nigeria - DPR
Norway DISKOS (NPD and the Diskos participants)
Oman OGDR (Ministry of Oil & Gas)
Peru NDC (PeruPetro)
South Africa Petroleum Agency SA
(http://www.petroleumagencysa.com/Data44.aspx)
Tanzania TPDC
Uganda
United Kingdom DataStore & DEAL (CDA)
USA US Geological Survey

Filename: NDR Models

Page 12

NDR Summaries (Incomplete a work-in-progress)

Filename: NDR Models

Cores

Reports

Modes and Models

Production

Joined

Mature
New
Mature
New
Mature
New
Mature
Mature

Raw

New
Mature
Mature
New
New
Mature
New
ITT
Proposed
New
New
Proposed
Mature
Mature
New
Mature
Mature

Field

Angola
W. Australia
Brazil
Cameroon
Canada (NS)
Colombia
Congo
India
Iraq
Kazakhstan
Kenya
Libya
Malaysia
Mexico
Mozambique
Netherlands
New Zealand
Nigeria
Norway
Oman
Peru
Qatar
South Africa
Uganda
UK (Offshore)
UK (Onshore)

Stage

Well Data

Pre-Stack

State

Post-Stack

Seismic

Appendix B

?
?
?
?
?

?
?

?
?

?
?

?
?

Data IPR

Funding

Private
Private?
State
Private
Private
State
State
State
Mixed
State
State
State
State
State
Private
Private

State
State
State
State
State
State
State
State
State
State
State
State
State
State
State
State

Private
State
State
State
State
State
Private
Private

State
State
State
State
State
State
Mixed
Private

Vendor
Schlumberger
Schlumberger
Landmark
Schlumberger
Schlumberger
Kadme

Landmark

Landmark
Various

Landmark
Landmark

Comments

Centralisation

Centralised model
2,000 TB

Distributed
Full
Full

204 wells
45 TB

Full
Full

NOC model
NOC model
NOC model, 40TB

tbd

600 TB; 5,000 wells. NOC Model (Petronas)

NOC model (PeruPetro)
105 wells; 60,000 km x 2D; 2,000 km2 x 3D
Predominantly physical data
130 TB
1st phase is seismic
Centralised model. Hardcopy & digital

Distributed?
Full
Distributed
Full
Full
Mostly
Full
Full
Full

Various

34 wells; 5,118km x 2D; 1,200 km2 x 3D

Distributed or virtual model
Charity

Scattered model
Full

Page 13

NDR Performance

Appendix C

The table below can be used to assesses the performance of an NDR against the criteria set out in 3 above.
Criteria
Regulations and Licence Conditions

NDR

Coherent, sensible and well-publicised regulations

Practical data standards (and in particular
standards for data exchange) which are prescribed
and enforced by the regulator;
Fair, effective and timely enforcement of the rules
Clarity on data ownership and associated rights
Material consequences for non-compliance.

Promotion and Facilitation of Oil and Gas Development

Global reach to encourage real interest from as
wide a field as possible
Simple access to relevant, quality data, published
at a fair price
Linkage to licence conditions and a Doing
Business in guide

Active Data Storage & Management

Knowledge of all relevant information available
Simple data access
Data usability (formats, media and completeness)
Acceptable costs and terms of use
Process alignment with the states policies and
arrangements for promotion
Process alignment with the states policies and
arrangements for long term preservation
Regulatory Compliance

Preservation of the National Legacy

A practical channel for archiving all relevant data
A pragmatic policy of data selection (you cannot
keep everything).

Filename: NDR Models

Page 14

Criteria

NDR

Secure storage conditions that are appropriate to

each of the data types stored (cores and digital
tapes require different conditions for example)
Assured evergreen preservation in a usable
format on a usable medium
Sustainable, long term funding

Other Local Conditions

Freedom of information
NOC access
Particulars of the licensing/permitting regime
Other requirements.

Filename: NDR Models

Page 15