Public Services International Research Unit (PSIRU) www.psiru.

org
Centre for Public Services www.centre.public.org.uk

Critique of the EC paper on Horizontal Evaluation of SGI*

by
David Hall (PSIRU, University of Greenwich) d.j.hall@gre.ac.uk ; and
Dexter Whitfield (Centre for Public Services) mail@centre-public.org.uk

25 September 2002

Commissioned by: EPSU

1 INTRODUCTION ..................................................................................................................................................... 2
2 CRITIQUE OF THE GENERAL APPROACH OF THE HE PAPER ............................................................... 2
2.1 THREE DIFFERENT TASKS: EVALUATION OF SGI PERFORMANCE, IMPACT ASSESSMENT OF DIRECTIVES,
MONITORING IMPLEMENTATION OF DIRECTIVES ............................................................................................................... 2
2.2 SOME CONFUSIONS .............................................................................................................................................. 3
2.2.1 Is liberalisation beyond evaluation? .............................................................................................................. 3
2.2.2 Decision-making process ............................................................................................................................... 4
2.2.3 Selection of sectors ......................................................................................................................................... 4
2.3 CHANGING REQUESTS: FROM SGI TO NETWORK INDUSTRIES .............................................................................. 4
2.4 DEMOCRATIC DEFICIT. ........................................................................................................................................ 5
3 CRITIQUE OF METHODOLOGY ........................................................................................................................ 5
3.1 BENCHMARKING.................................................................................................................................................. 5
3.2 COMPARABILITY. ................................................................................................................................................ 6
3.2.1 Relevance of indicators .................................................................................................................................. 6
3.3 INADEQUATE IMPACT ASSESSMENT ..................................................................................................................... 7
3.4 CROSS SECTOR ANALYSIS .................................................................................................................................... 7
3.5 CONSULTATION WITH USERS AND CONSUMERS ................................................................................................... 8
3.6 NO CONSULTATIONS WITH STAFF ........................................................................................................................ 8
4 CONCLUSIONS AND RECOMMENDATIONS .................................................................................................. 9
4.1 ASSESSMENT OF PERFORMANCE OF SGI .............................................................................................................. 9
4.2 IMPACT ASSESSMENT OF LIBERALISATION DIRECTIVES ....................................................................................... 9
4.3 WIDER SOCIAL AND EMPLOYMENT DATA ............................................................................................................ 9
4.4 MAKE DEMOCRATIC EVALUATION CENTRAL ..................................................................................................... 10

*
A Methodological Note for the Horizontal Evaluation of Services of General Economic Interest
Brussels, 18.6.2002 COM(2002) 331 final
http://europa.eu.int/comm/economy_finance/publications/structural_policies/2002/com_2002_331_en.pdf

Public Services International Research Unit (PSIRU)

School of Computing and Mathematical Sciences, University of Greenwich, Park Row, London SE10 9LS U.K.
Email: psiru@psiru.org Website: www.psiru.org
Tel: +44-(0)208-331-9933 Fax: +44 (0)208-331-8665

Centre for Public Services (CPS)

1, Sidney Street, Sheffield S1 4RG.
Email: mail@centre-public.org.uk Website: www.public.service.org.uk
Tel: +44 (0) 114 272 6683 Fax: +44 (0)114 272 7066
 PSIRU / CPS www.psiru.org / www.centre.public.org.uk

1 Introduction
This report is a critical commentary on the EC paper “A Methodological Note for the Horizontal Evaluation
of Services of General Economic Interest” (HE)1 It includes reference to the commission’s earlier report
“Market performance of network industries providing services of general interest: a first horizontal
assessment” (MPNI) 2That report was in effect a prototype of the kind of evaluation the main report is
proposing.

The HE paper is a confused and confusing document. It refers to a number of requests from European
Councils and the parliament for evaluation of the impact of services of general interest – a wide remit – but
then proposes a methodology for annual horizontal evaluation of a number of network industries – a
narrower remit. These include electricity, gas, telecoms, post, rail, air transport, passenger road transport; but
exclude water, broadcasting, waste management; and also excludes other public services such as health,
education, social services, justice.

The stated objectives of the HE paper vary in different places. They include both narrow assessments of the
extent to which competition has been introduced, but also, in a number of places, assessing the impact of
liberalisation on services; as well as the more general assessment of the value of SGI to the EU.

The paper refers to commitments to public debate and use of democratic forums of the EU, and consultation
with stakeholders. But the methodology proposed includes only surveys of consumer opinions and ad hoc
consultation of stakeholders on selected issues.

This report provides a critical commentary on the framework of the paper and on the methodology
proposed. The final section suggests a response to the paper with proposals which attempt to build on the
positive issues which were included in the original demands:
- the principle of regular EU-wide evaluation of the contribution made by SGI
- an assessment of the impact of liberalisation required by directives on some SGEI
- basing this process on democratic debate and the participation of stakeholders.

2 Critique of the general approach of the HE paper

2.1 Three different tasks: evaluation of SGI performance, impact assessment of
directives, monitoring implementation of directives
A central problem with the paper is its confusion between three different projects:

• monitoring and evaluating the delivery of services of general interest (SGI) across the
community, to see how far the community objectives for SGI are being fulfilled. This would
examine all SGI, not just those for which there are liberalisation directives. It would examine their
contribution not only to the narrow role of SGEI mentioned in the Treaty, but also their contribution
to broad Treaty objectives as set out in for example Article 3 on “the strengthening of economic
and social cohesion”, “a policy in the sphere of the environment”, “the establishment and
development of transEuropean networks”, “the attainment of a high level of health
protection”, “education and training of quality and … the flowering of the cultures of the
Member States”, “consumer protection”, “promote jointly [in developing countries]
economic and social development”; and “the Community shall aim to eliminate inequalities,
and to promote equality, between men and women”. 3 The HE paper fails to develop this
approach at all – there is no mention of equality anywhere, for example.

29 April 2002 Page 2 of 10

 PSIRU / CPS www.psiru.org / www.centre.public.org.uk

• an impact assessment (IA) of the EC directives on liberalisation of network industries such as

electricity, gas, telecomms, rail. This would examine the effect of the directives on other community
policies such as social inclusion, equality, employment, sustainable development etc , and draw
conclusions about the revision or otherwise of the directives. Such an exercise would not cover the
sectors for which there are no directives requiring liberalisation – water, health, education etc. The
HE paper is extremely confused on whether it wants to engage in this kind of impact assessment at
all or to refuse to do so on principle, as the MPNI report did (see below).

• monitoring of the implementation of the directives on network industries: for example,

monitoring the extent to which energy markets have been opened in each country. This is a more
limited exercise, which does not attempt to evaluate the impact of the directives but simply the
extent to which they are being implemented by member states – and is already being carried out by
the EC under each directive (e.g. the Electricity directive). It therefore seems unnecessary to
duplicate this activity.

The first two tasks are separate but both worth carrying out: however, the HE paper fails to propose a
satisfactory methodology for either. The third one, monitoring the extent of the implementation of
directives – is already being adequately carried out, and it is not at all clear how it is enhanced by
incorporation in a horizontal exercise.

2.2 Some confusions

The HE proposals suffer from a number of confusions, most of which are attributable to the failure to
distinguish these objectives, and an underlying lack of enthusiasm for doing what was requested by the EP
and the Nice and Laeken councils.

2.2.1 Is liberalisation beyond evaluation?

The HE paper is utterly misleading on whether the evaluation concerns the liberalisation policies of the
directives themselves. The paper states that it should address “whether changes in competition have had a
positive impact on the performance of the network industries in terms of productivity, innovation,
employment, or growth, but also on the equally important aspects of affordability, quality and accessibility
of services of general interest.” (2.2, p.4). However, the proposed report on analysis of structural changes
refers only to “whether there is active competition between suppliers….whether there are barriers to entry.”
(4.1.i, p.6) , but this does not address the impact of the liberalisation itself, and then refers to “market
performance” and how this impacts on services: but this is quite different from the impact of the
liberalisation requirement. There is one reference, as a note to the proposed surveys of consumer
satisfaction, that “special attention will be paid to the impact of market opening on those consumers most
sensitive to changes in the provision of SGI” (4.2.ii, p.7), but there is no hint how this will be done and no
explanation as to why such a central issue should only feature in the interpretation of one element of the
proposed evaluation. The impact of market opening is relegated to a catch-all, ungrammatical sentence
about a report on horizontal topics relevant to most services: “It will address issues such as the market
opening and reform of network industries and employment, their integration at European level, their
environmental impact, productivity and technological change, or their impact on social and territorial
cohesion.” (4.3, p.8). But there is no methodology proposed for this.

The view of the Commission may be more clearly inferred from the December 2001 MPNI report, which
has a clear statement that the impact of liberalisation is not and cannot be evaluated : “The report is not
meant to evaluate the process of opening to competition and liberalisation in different sectors initiated either
at the EU level or the national level.” (MPNI p.5) It justifies this with the assertion that it is too soon:
“Liberalisation’s full impact cannot be measured or evaluated yet. More experience of the operation of these
markets under more competitive conditions is needed to fully assess liberalisation.” (ibid).

29 April 2002 Page 3 of 10

 PSIRU / CPS www.psiru.org / www.centre.public.org.uk

If this position remains unchanged, liberalisation of SGEI is elevated to the status of a belief which cannot be
examined or subjected to testing by examining evidence. It is not plausible to claim that it is “too early”: a
number of member states have many years experience of liberalisation in these sectors, and lessons can and
should be drawn from this experience.
2.2.2 Decision-making process
The HE paper states that “It seeks to make a positive contribution to the decision making process in all
policy areas related to network industries providing services of general interest by providing a methodology
that can help produce precise, thorough and comparative evaluations of the performance of those network
industries and the services they provide” (s.2, p.3), but does not make explicit which decisions it expects to
be influenced. The paper is silent on who receives the reports, what decisions they will be used to review,
who is involved in this review process, and how that review process might work . The position of social
partners for example is not mentioned, except implicitly in the reference in to the preparation of the third
section of the reports, for which experts will be commissioned, where “on an ‘ad hoc’ basis, relevant
stakeholders may be consulted” (4.3, p.8).

In this respect the December 2001 MPNI implies that some stakeholders are the subject of the evaluation
process, not participants. On employment in energy, for example, it refers to reports and data on large scale
job losses, then refers to the trade union position and rejects it (section 1.5, p.28). In effect, the MPNI was
used not to evaluate the EC policy but to evaluate the EPSU position.
2.2.3 Selection of sectors
The selection of sectors is not clearly justified. In particular, there is no reason given for the omission of
water and sanitation. Water is a sector on which there are major EC directives (most recently the WFD), and
also a sector with considerable economic, environmental and social impacts. In the December 2001 MPNI
water is not covered in the appraisals of market performance or public service obligations, but there is a
section on consumer satisfaction in water which covers 15 countries. These do not include France – a
remarkable omission since it is the home of the dominant water companies, and a country where experience
of privatized water is both extensive and highly controversial. The MPNI paper gives no reason either for the
inclusion of water in this section only, or for the exclusion of France from the water survey. The limitation
to network industries makes sense if the objective is an impact assessment of the liberalisation directives, or
even their implementation – but not for the more general objective of assessing the performance of SGI.

2.3 Changing requests: from SGI to network industries

As the HE paper states, the Commission has been requested to propose work in this area by a number of EU
council meetings, as well as by the European parliament. However, the paper has imposed a different and
narrower perspective, because its own proposals for monitoring of network industries are far narrower than
the original requests - which wanted to see a much broader evaluation of SGI which focuses on the quality
of services and their social impact.

The earliest request mentioned, from the Nice Council, was very general: it covered all SGI, and asked that
the evaluation should cover their performance in general “particularly in terms of quality of service,
accessibility, safety and fair and transparent pricing…. ” (as quoted in MPNI, Introduction, p.5). It was not
restricted to network industries, and not restricted to those sectors covered by liberalising directives.

The European parliament too was clear that its was concerned with the impact of liberalisation being
introduced into SGI , and the proper functioning of SGI themselves: their response, in October 2001, to the
EC report on SGI “Calls on the Commission to perform without delay a precise and comparative evaluation
of the real impact of the policy of liberalisation of services of general interest before embarking upon further
liberalisation”: it also went on to approve the use of the Cardiff process “as a framework for regular
evaluation of the functioning of services of general economic interest within the single market” .4

The Laeken council conclusions on SGI also envisaged wider coverage than just network industries. It
supported “the need for effective and dynamic evaluation at Community level of the competition effects and
performance of services of general interest.”. It also expected that the evaluation would cover a range of

29 April 2002 Page 4 of 10

 PSIRU / CPS www.psiru.org / www.centre.public.org.uk

issues, including an assessment of public service obligations: “market structure and performance, including
employment aspects, an economic and social assessment of public service obligations, and citizens' and
consumers' opinions on the performance of services of general interest and the impact of liberalisation on
them” and welcomed the Commission’s intention to produce an evaluation methodology. 5

The restriction to network industries appears to have been introduced by the Commission itself when it
carried out the MPNI assessment in December 2001 as an annexe to the report on product and capital
markets. (see above). This was then subsequently welcomed by the Barcelona council, which asked for
further reports: “following the first horizontal assessment of the market performance of network industries
providing services of general interest presented by the Commission in the framework of the Cardiff process,
the Commission is invited to present further assessments”6

The HE paper thus appears as the latest stage in a process whereby the Commission is restrictively distorting
the requests of representative institutions to fit its own preferred framework. In the process, the evaluation of
the contribution of SGI, which has been the central element in these requests, has been marginalised.

2.4 Democratic deficit.

The paper states that the evaluation will be based on “feedback from citizens, consumers and other
stakeholders.” (2, p. 4) and that the “methodology has to take into account the opinion of citizens, as
consumers and stakeholders,” ((3.1.(d), page 5). It also promises that “a permanent mechanism for the
monitoring of citizens’ opinion and their evolution will be established” (ibid.); endorses the European
Parliament’s view that “ppublic participation could be greatly expanded”, and the EP’s proposal to
“organise the debate within the various existing forums (Economic and Social Committee, Committee of the
Regions, consultative bodies, associations involved in services of general interest initiatives and consumer
associations”, and the HE paper adds that “The results of this debate should be taken into account and
provide guidance for the annual horizontal evaluation, and the evaluation should itself be the subject of
debate”(ibid.).
However, the methodology actually proposed falls far short of these democratic aspirations. The role of
citizens and stakeholders falls out of the picture, and this aspect of evaluation becomes exclusively
concerned with Europeans as consumers. The three key questions include one on user perception, but this
refers solely to “consumers perceptions of market performance”, and promises to “undertake consumer
satisfaction surveys” (p.4).

Assessments by representative and democratic organisations, by contrast, are marginalized. Stakeholders

are restricted to being consulted “on an adhoc basis for specific issues” (p.5), and the debate organized by the
EP will be ‘taken into account’ (ibid.). There is no role for the sector social dialogue committees, for
example, and there is no acknowledgement of the ETUC/CEEP proposal for a European-wide ‘observatory’
on SGI . Although the paper claims that there will be a “permanent consumer consultation process” (4.2, p.
7) this turns out to consist of consumers being the passive subject of market research – the EC will
“regularly produce indicators measuring consumers’ satisfaction” and “will also conduct ad-hoc surveys to
gather more information”. (ibid.). The Annex sets out indicators of consumer perception of services, none
at all of citizen or stakeholder views.
This approach completely fails to deliver the paper’s promise of “feedback from citizens, consumers and
other stakeholders.” There is an opportunity here, and a demand from the parliament, to design a
participatory process for evaluating the importance of SGI, but the paper fails to deliver anything resembling
this.

3 Critique of methodology
3.1 Benchmarking
The scope of European-wide evaluation imposes certain practical constraints such as the reliance on
benchmarking of performance indicators. It is important to remember that this process relies on data
benchmarking which is the basic methodology for comparing performance between organisations. The more

29 April 2002 Page 5 of 10

 PSIRU / CPS www.psiru.org / www.centre.public.org.uk

detailed types of benchmarking - process, functional and strategic benchmarking –provide a more
comprehensive and detailed comparison of performance but are unlikely to be feasible on a European level
except in specific sector studies.

Reference is made to providing a methodology which is "precise, thorough and comparative" but there are
substantive questions about whether this is currently technically feasible given the limitations on data
availability. It may only be possible if excessively large resources are diverted into benchmarking and
monitoring. Reference is also made to expanding the first list of indicators, and consultation of ‘relevant
stakeholders’ – but it is necessary to be far more precise about who is involved and what they are consulted
about and how the process will be related to democratic mechanisms.

3.2 Comparability.
Benchmarking performance indicators at a European level raises certain difficulties of comparability because
of:

• Variations in the basis on which information is collected and different interpretations of performance
between nation states and sectors;
• Difficulties in correlating performance indicators used in benchmarking with actual consequences.
• Performance can be distorted by selecting, grouping or separating indicators.
• Social and public interest obligations imposed by nation states makes comparability between service
providers more complex.
Emphasis on quantitative performance indicators which often marginalises qualitative information about
performance. These points are elaborated in the next sections.

There is a need to ensure:

• comparability of timescales;
• use only verifiable, preferably audited, performance information which is sourced and dated;
• subjective issues and non-quantifiable objectives, particularly those relating to corporate policies
(differences between public and private sectors), should be included in benchmarking;
• differences in standards, geography, cultural, social and economic conditions and other variables are
taken into account.
• Differences in regulatory regimes are taken into account;
3.2.1 Relevance of indicators
Some of the proposed indicators are of highly questionable relevance. For example, the Accessibility section
of the Annex proposes as an indicator – "number of post offices/number of letter boxes per inhabitant". This
is virtually meaningless at a national level and has even less relevance at a European level because it does
not take account of local and regional variations between urban and rural areas. A high ratio of post
offices/letter boxes per inhabitant does not mean that people in rural areas are well or equally served: high
levels in urban areas can mask poor service in rural areas. The ratio would simply measure urbanisation.

The focus of the report is on the functioning of markets yet it is highly questionable whether performance
indicators alone can be used to identify misallocation and/or malfunctioning of markets. Performance
indicators do not always adequately reflect market conditions - for example the extent to which electricity
markets have been opened may serve to obscure the degree of market domination by a few producers in any
one country and the EU as a whole.
There are further difficulties, for example, in comparing air and rail transport because of the fragmentation of
providers and different levels of provision. For example, rail services in Britain are provided through 24
private franchises making a national average less then useful in comparison with rail services in some other
European countries. Few relate to regional boundaries. Benchmarking in these circumstances is likely to be
general and is likely to mask any inter/intra regional variations, which may hold more significant lessons.

29 April 2002 Page 6 of 10

 PSIRU / CPS www.psiru.org / www.centre.public.org.uk

3.3 Inadequate impact assessment

Despite repeated claims to evaluate ‘impacts’ there is no clear methodology for an impact assessment which
examines the effect of policies on services, users, staff, markets. (This may be another consequence of
confusing impact assessment of the effect of the directives with analysing the overall performance of the
service or the extent to which the directives have been implemented by member states.). The proposed
methodology excludes evaluation of , for example:
• Quality of employment (and the broader impact on the labour market), for example of the:
o Effect on terms and conditions of existing staff.
o Impact of outsourcing on different services and grades.
o Creation of two-tier workforce by changes to terms and conditions of new staff.
o Provision of pensions.
o Changes in quality/quantity of training provision by employers.
o Changes in geographic provision of training relative to unemployment and socio-economic
needs
o Changes in employment of equality groups (gender, age, disability, race etc) within the
sector.
o Ratio of trainees from different equality groups.
o Implementation of policies to promote diversity.
• Equalities (in service delivery and employment)
o Service provision relative to social need (note: to assess "the number of passenger seats per
inhabitant -trains, airlines" - as in the Methodological Note is meaningless)
o Social groups and industries affected by breaks or delays in service.
o Equity in different tariffs and costs and how this impacts on different groups.
o Equity in payment systems for the service.
o Cost of facilitating access (additional costs incurred in gaining access - transport, car
parking, additional in-service costs, for example, high cost/low quality catering).
o Social relations between staff and users.
• Social Inclusion
• Accountability
• Transparency and disclosure

3.4 Cross sector analysis

The report promises a cross-sectoral review of horizontal topics with reference to "….market opening
….environmental impact, productivity and technological change and impact on social and territorial
cohesion" (4.3, p8). The value of this approach is questionable for the following methodological reasons:
• Comparability: although the broad the reform of network industries follows certain principles, the
application and impact in each sector is distinctly different. It is highly questionable whether the
comparison of environmental impact between, for example, the electricity and postal service can be
productive.
• There are limitations in data benchmarking within and across nation states. These will be
compounded in a horizontal evaluation of seven sectors across 15 countries. The value of
‘evaluation’ of very different network services, implementing reform in different ways, timetables
and so on must be questionable.

The ability to understand cause and effect, a key function of evaluation, and essential to formulating lessons
learnt, is also constrained by the data benchmarking. E.g. If the data examined does not include information
on ownership of energy, waste, water, telecoms and transport services, some important causes and effects
will be systematically obscured. Sector studies

It is recommended that the evaluation of the performance of SGI should focus primarily on more
comprehensive and detailed sector studies. In turn the sector studies should be able to provide horizontal
evaluations with a summary of the core findings. Much of the information in the Annex should be sourced

29 April 2002 Page 7 of 10

 PSIRU / CPS www.psiru.org / www.centre.public.org.uk

from fuller sector studies rather than horizontal evaluations having to rely on new benchmarking exercises or
data collection.

3.5 Consultation with users and consumers

The importance of consumer consultation is emphasised, but the proposed methodology falls far short of the
paper’s democratic aspirations (see above). Reference is made to two methodologies - indicators measuring
consumer’s satisfaction based on panel data, and ad-hoc surveys. However, there are a number of major
reservations about how horizontal evaluation can realistically take account of user views in a meaningful
way.

• Which users? There is a range of different types and classes of users in the network services. These
reflect different economic, social and political needs and interests. E.g. large commercial users of
electricity are well placed to influence debate and use their market power; the Commission needs to
ensure that the interests of small consumers on low incomes are heard
• Selection of consumer organisations: Who will select the consumer organisations and how
representative will they be e.g. just representative of middle income groups with spending power?
Will these be confined to the consultation panels and/or organisations already established (frequently
by government and utility companies)?
• Which organisations will be consulted in rural areas where there is rarely a specific service users
group?
• Non-users and barriers to access: Non-users and the reasons for non-use (which could include a host
of reasons such as accessibility, price, availability and so on) are very important but the proposed
focus of the horizontal evaluation appears to be on ‘satisfaction of use’ i.e. covering only those
currently using the service.
• Interpretation of customer surveys: A footnote states that "special care will be taken in the
interpretation of customer surveys", but no further detail is given.
• Scope of customer perceptions: The Annex refers to levels of satisfaction on the overall service,
price and quality, information problems between consumers and providers, indicators of ‘conflict
resolution’ and difficulties in switching providers. These are all related to market mechanisms and
have little to do with social and public policy matters, e.g. achieving full employment, improving
social cohesion, reducing inequality.

Taking all these factors into account indicates the difficulty in European-wide customer satisfaction analysis.
The HE paper acknowledges this when it promises to take ‘special care’ in the analysis of user satisfaction:
but this should be a normal practice, what is needed is to spell out a methodology which will achieve deliver
a multi-faceted assessment of the diversity of users. .

For example, the use of city-wide citizens panels and market research in Britain under the Best Value regime
in local government has highlighted a number of weaknesses. Electoral representation does not always
equate to representation of service users: the quality of user views in market research and broadly based
‘satisfaction surveys’ is generally superficial; little real qualitative intelligence is gained about user views
and preferences through these methods; and market research is often used to justify decisions by local
authorities which conflict with the express views and needs of particular communities.

The horizontal evaluation methodology also states that "…..consumers behaviour will be closely surveyed"
(4.1 (i) page 6. However, this is a separate issue from surveying consumers’ views. Identifying and
understanding consumer behaviour, particularly cause and effect, is difficult enough at the local community
and national levels, let alone at European level. The potential for mis-use of user views is wide.

3.6 No consultations with staff

There is a final, glaring omission in the paper – nowhere does it provide for consultation with staff. The
workers in a service, especially public services, are an extremely important source of informed, often critical,
appraisal of the performance of the service and the impact of external requirements. Evaluation of these
services, and the impact of liberalisation, should involve a system of consultation with staff at local, national

29 April 2002 Page 8 of 10

 PSIRU / CPS www.psiru.org / www.centre.public.org.uk

and European level. The evaluation process should use existing machinery at all levels, and/or include
special processes – again, there is the opportunity to explore the potential of participative processes used by
some municipalities in the EU.

4 Conclusions and recommendations

The foregoing critique can provide a basis for a response to the paper, relevant to stated objectives of the
paper itself, including the following elements.

4.1 Assessment of performance of SGI

The central role of the HE should be to provide an assessment of the performance and impact of the SGI
themselves. This was the original stated objective of the Nice and Laeken councils, and the European
Parliament, and should be made the central objective, not mixed with others or replaced by a different
agenda preferred by the Commission.

Such an assessment should cover all SGI, not only network industries, and so should include for example
water, waste management and other SGEI. The assessment of health and education could also be attempted
in principle, but may pose special difficulties of their own. However their exclusion should be argued, not
assumed.

This assessment should also concern a full range of economic and social contributions of SGI, evaluated
against the treaty provisions on SGI, and other treaty aims e.g. as stated in articles 2 and 3, as well as the
general objectives of the services themselves, their importance to economic activity, and the full range of
social impacts, including equality, quality of employment etc..

4.2 Impact assessment of liberalisation directives

The HE should also include the distinct task of impact assessment of existing directives introducing
liberalisation into SGI. This was another clear intention of the original requests. Liberalisation should not be
exempt from public examination as it was in the MPNI (“It is too early…etc” : see above). It should be
clearly subject to a full and rigorous impact analysis, as is promised for all future policies, projects or
programmes, including new Directives, in the policy making process (in the EC paper Action plan
"Simplifying and improving the regulatory environment" Brussels, 5.6.2002 COM(2002) 278 final).

The impact assessment should not be confused with the routine measurement of the implementation of the
directives, e.g. the proportion of market opening, which is already carried out. The HE should ensure that
administrative information about market opening in compliance with directives, for example, is not confused
with real data on the extent of effective competition and consumer choice: 100% market opening may
coexist with very little competition or choice.

Finally, this impact assessment must include explicit counter-factuals, so that it is clearly evaluated in terms
of what would have happened in the absence of these directives, or if alternative directives had been adopted
(for example, requiring decentralisation and maximum subsidiarity of policy making on energy structure,
without centrally defined requirements on the role of the market). These counter-factuals could also include
reference to international experience and experience over time, so that the process could incorporate learning
from the past experience of European nations and the practices of others.

4.3 Wider social and employment data

Both the performance assessment of SGI and the impact assessment of the effects of liberalisation need to be
wide ones which integrate social, employment and labour market, equality and environmental with economic
assessment. There is a need for a much wider range of data to be examined, concerning the distribution of the
impact of energy liberalisation (fuel prices for different groups), accountability mechanisms (e.g. rail
contracting), employment-related data (energy), contract changes (e.g. water).

29 April 2002 Page 9 of 10

 PSIRU / CPS www.psiru.org / www.centre.public.org.uk

4.4 Make democratic evaluation central

The paper’s proposals do not live up to its declared aim of creating democratic debate. Both evaluation of
SGI and impact assessment of the liberalisation directives should be clearly structured around a democratic
process, whose elements are mentioned but not used in the HE report:

• an organised public debate within the various existing forums – it is important that there is
widespread debate especially on SGI because the values used to evaluate the performance of the
services, and the impact of the directives, are values based on democratic organisation not on market
forces
• a permanent mechanism for the monitoring of citizens’ opinion – which must be far more than
passive responses to market research of consumer satisfaction. There should be a clear process which
requires annual submission of questions on SGI to specific bodies and stakeholders, and provision
for public meetings and debate in member states. The ETUC/CEEP proposal for a European
observatory on SGI should be taken up as part of this.the debate should provide guidance for the
annual horizontal evaluation, and the evaluation should itself be the subject of debate – the
outcomes of these debates need to be reported and collated and used to drive the HE processes. This
is very different from market research on consumer perceptions.
• Encouraging the expression and contribution of staff views to the process should be an explicit
element in consultations.

1
Brussels, 18.6.2002. COM(2002) 331 final.
http://europa.eu.int/comm/economy_finance/publications/structural_policies/2002/com_2002_331_en.pdf
2
Brussels, 7/12/2001: Annex to the report on the functioning of product and capital markets. SEC (2001) 1998).
http://europa.eu.int/comm/internal_market/en/update/economicreform/cardiff02enfull.pdf
3
Consolidated version of the treaty establishing the European Community
Official Journal C 340, 10.11.1997, pp. 173-308
http://europa.eu.int/eur-lex/en/treaties/dat/ec_cons_treaty_en.pdf
4
(European Parliament resolution on the Commission communication, ‘Services of General Interest in Europe’, points
2-3; in A5-0361/2001 17 October 2001. REPORT. on the Commission communication ‘Services of General Interest in
Europe’ (COM(2000) 580 – C5-0399/2001 – 2001/2157(COS)) Committee on Economic and Monetary Affairs)
5
(Conclusions on services of general interest (from Internal Market, Consumer Affairs and Tourism Council to Laeken
European Council) Press Release: Brussels (4/12/2001) Nr: 14866/01
http://ue.eu.int/Newsroom/LoadDoc.asp?BID=75&DID=68798&LANG=1 )
6
(Barcelona council Conclusions, no.18, p.56).

29 April 2002 Page 10 of 10