Differences in the Game Specific Skills of Elite and Sub-Elite Youth

Football Players: Implications for Talent Identification

Mark Waldron and Paul Worsfold

Department of Sport and Exercise Sciences, University of Chester, Parkgate Road,

Chester, CH1 4BJ, UK

Abstract

The concept of multidimensional talent modelling offers a recent attempt to

objectively identify sporting talent in adolescents, of which game specific skill
assessment has become a significant dimension. Previous studies have used
closed skill testing to assess ‘open’ skills in youth football players, often
finding such measures to discriminate between elite and sub-elite groups. The
current study used an example performance analysis template to; 1) Identify
differences in performance indicators between pre-determined groups of elite
and sub-elite performers during an open match environment; 2) Identify
differences in performance indicators between positional groups of elite and
sub-elite performers at separate levels of competition; 3) Individually identify
sub-elite players compared to a normative profile of elite positional
counterparts. Results from Mann-Whitney U testing suggested that elite
players are significantly higher (p<0.0028) performers within 9 of 18
performance indicators. Comparisons refined by position and round of
competition found only one indicator to distinguish between elite and sub-
elite players. The use of a normative profiling method demonstrated how sub-
elite players may be analysed relative to elite counterparts. Results are
discussed in relation to the advantages of performance analysis as a preferred
measure of game specific skills in the talent identification process.

Key Words: Talent Identification; Youth Football; Skill Assessment

1. Introduction

The early identification of young talented sports people has become increasingly
important across many performance domains (Abbott et al., 2005), most notably in the
popular leading sports such as football. However, against a backdrop of an ever-
increasing standard of elite performers (Martindale et al., 2007), concerns over Britain‟s
ability to produce localised talent, particularly within the sport of football, have been
raised by academics (Reilly et al., 2000b). Consequently, the identification and
development of sporting talent has become a research area of significant interest for sport
scientists (Williams and Reilly, 2000).

9
In order to select suitable candidates for talent development programmes, research has
adopted a multi-dimensional approach. This often includes objective measurements of
elements such as anthropometric, physiological, psychological and game specific skill
variables as a method of identifying criterion values to represent elite youth populations
(Gabbett et al., 2007; Vaeyens et al., 2006; Elferink-Gemser et al., 2004; Falk et al.,
2004; Reilly et al., 2000a; Reilly et al., 2000b; Pienaar et al., 1998). These measures are
subsequently advocated for use within longitudinally designed research in order to
distinguish between elite and sub-elite samples, thus identifying the potential
determinants or criterion levels of elite youth performance. For the present study,
particular interest was placed upon the dimension of game specific skill assessment in
football and how this may be improved for future use within longitudinal, multi-
dimensional talent identification processes.

From the perspective of game specific skills in team sports, research has reported fine
motor skill, such as close control dribbling, to occur most frequently as the significant
difference between pre-determined elite and sub-elite performers (Pearson et al., 2006).
For example, studies in youth football (Huijgen et al., 2009; Vaeyens et al., 2006; Reilly
et al., 2000b), hockey (Elferink-Gemser et al., 2004), water polo (Falk et al., 2004) and
volleyball (Gabbett et al., 2007) have each provided evidence to support the notion of
skill testing as a discriminating factor between elite and sub-elite playing levels, using
additional gross motor skills such as passing or shooting within the respective test
batteries. Within the aforementioned research, closed skill protocols such as „slalom
dribbles‟ (Vaeyens et al., 2006) which have been adapted from previous, and somewhat
dated studies (Reilly and Holmes, 1983) are often used to assess the skill of a player
based upon preconceived outcome measures (i.e. time to complete a given course of
cones). However, although the use of closed skill testing appears to offer a clear value for
talent identification processes, the continued use of such tests as part of a scientific
identification template is questionable. This is that, it remains the implicit assumption
within current research that closed skill testing can provide a valid assessment of game
specific skill in sports which are open by nature, yet the inability of closed skill tests to
transfer from a contrived lab-based environment to an open game situation has long been
raised as a growing concern in academic research, where ecological validity remains an
aspiration (Vaeyens et al., 2008; Falk et al., 2004; Van Rossum and Wijbenga, 1993).

The methods used within previous studies, such as Vaeyens et al. (2006) and Reilly et al.
(2000b), are also subject to further criticism based on the ignorance of playing position as
a potential confounding variable. James et al. (2005) recently reported the significant
impact of playing position on the in-game skills or „performance indicators‟ (Hughes and
Bartlett, 2002) demonstrated by elite rugby union players. For example, the „prop‟
positional cluster shown a median of 3.02 successful tackles (confidence limit = 4 and
2.85), opposed to the „openside flanker‟ group, where a median of 10.59 (confidence
limit = 14 and 9) was demonstrated. Gaining an understanding of positional differences in
sport has many obvious advantages, not least the specificity of training demands and
individual recovery strategies. The same is true for talent identification research, where a
further notable limitation appears to lie in the absence of key game skills, such as tackling
and interceptions, which play a perpetually significant role in the modern game of

10
football, particularly for defensive positional groups. Such measures, however, remain
excluded, often by nature, from closed skill testing protocols. Furthermore, the lab setting
is far removed from the competitive environment which has been noted as key to talent
identification systems (Hammond, 2001; Kozel, 1996). Consequently, there is no
measure to control maximal performance output or a consistency of performance output
between performers in previous research.

A key aim of the current paper was not only to describe differences in game specific skill
variables between „elite‟ and sub-elite players but also to attempt to identify initial higher
levels of game specific skill in individual sub-elite players relative to elite counterparts.
In the present day, the concept of “normative profiling” (Hughes et al., 2004, pg. 205), a
method designed to analyse potential patterns in performance indicators derived from a
team or individual analysis, ostensibly offering some prediction of future performance
(James et al., 2005; Hughes et al., 2001), appears to offer a more suitable basis with
which analyse and evaluate player behaviours. A technique suggested by O‟Donoghue
(2005) offers a method of profiling which reports the typical performance and associated
variance or „consistency‟ of an individual or team, based on a pre-chosen group of
performance indicators. As such, O‟Donoghue‟s method holds many advantages over
previous approaches (Hughes, Evans and Wells, 2001), most notably, that a typical
performance can be related to normative percentile ranges of any chosen data set.

The advantages of such a method seem apparent for talent identification purposes, since
the performance and, indeed, the long term consistency of performance in an apparently
„sub-elite‟ player (based on current status) can be compared to elite groups. However,
this method assumes a contextual consistency in which to measure performance of both
elite and sub-elite players, a precautionary suggestion that O‟Donoghue (2005) raised for
analysts attempting to normatively profile, exampled by the influence of court surface on
tennis performance. As such, both elite and sub-elite players are required to share a
playing environment in order to deploy a comparison of groups that may inform a valid
talent identification process.

An exemplar generic framework is accordingly presented, attempting to identify game

specific skill in youth footballers. This involved the use of performance analysis
processes to provide an objective and ecologically valid protocol to potentially
supplement current multi-dimensional talent identification models. The protocol was
embedded within an extra-curricular national cup competition at under 14 level (Key
Stage 3) as a way of sourcing a broad base of both externally pre-determined elite and
sub-elite players for analysis. The use of the system was directed towards identifying
performance indicators that distinguish between ability levels and to identify sub-elite
players with potential elite skill characteristics. Specifically, the objectives were to;

1) Identify differences in performance indicators between pre-determined groups of

elite and sub-elite performers during an open match environment.

2) Identify differences in performance indicators between positional groups of elite

and sub-elite performers at separate levels of competition.

11
 3) Individually identify sub-elite players based against the normative profile of elite
positional counterparts.

2. Method

2.1. Ethical and Legal Clearance

Ethical approval was granted by the University of Chester Ethics Committee,
supplemented by further approval of the English Schools Football Association to work
within an educational domain. Child protection procedures and all photography and
filming guidelines were adhered to throughout. This also included CRB disclosure for the
lead researcher. Informed consents were completed by the participants and the
parent/guardians, who were assured of complete confidentiality and anonymity.

2.2. Design and Participants

In accordance with Hammond (2001) and Kozel (1996), the seven schools who opted to
participate in the study (n=7) were analysed within a progressively competitive
environment based on geographical filtering. Schools were designated with a number
from 1 to 7 and are referred to this for the remainder of the study. Figure 1 shows a
structural design of the study.

School 1 vs. School 3 vs. School 5 vs.

th
4 Round School 2 (Game School 4 (Game School 6 (Game
1) 2) 3)

School 2 vs. School 3 vs.

5th Round School 7 (Game School 5 (Game
5) 4)

Figure 1. Structure of analysis

Ninety eight male pupils (n=98) at KS3 chose to participate in the study. Seventy one
participants (n=71) were analysed (exclusion of goalkeepers and substitutes playing less
than 1 half during research). Eighteen players were pre-determined elite players (n=18),
and fifty three were pre-determined sub-elite players (n=53). This was determined by
players who are currently contracted at football clubs, bearing Academy/School of
Excellence status (Reilly et al., 2000b).

12
As a result of only three school teams participating over two rounds and the exclusion of
goal keepers, thirty players (n=30) were analysed over two rounds of competition. Each
sub-elite player was given an identification number for individual reference throughout
the study. Sub-elite players in the 4th round were coded with the letter „A‟ before the
identification number, players in the 5th round were coded with the letter „B‟ before the
identification number. Elite players did not require a coded number.

2.3. Analysis system and Identification of Performance Indicators

Three separate hand notation systems in the form of frequency charts were used
(defender, midfielder and attacker), all of which were inclusive of 18 performance
variables. A definition dictionary was used to maintain a consistency whilst analysing
data (James, 2006). Each definition was used or adapted from previous research (see
Rowlinson and O‟Donoghue, 2008; Tenga and Larsen, 2003; James, Mellalieu and
Hollely, 2002) and viewed as a generic measure of performance, suitable for the current
example model. All of the player‟s scores were recorded as frequencies. To support the
application of long or short passing occurrences, a schematic pitch representation was
used (Hughes and Franks, 2004), separating the pitch into nine equal zones. An example
of a short pass successful definition is provided below;

“When a player attempts to play the ball with any part of the body to one of his team-
mates to allow that player possession of the ball. The ball must not travel beyond the
distance of a pre-designated zone on the schematic pitch representation and must be
controllable by the receiving player, causing no break in their movement or momentum”.

Table.1. Coded abbreviation for use in the hand notation system

Code Indicator
SPS Short pass successful
SPU Short pass unsuccessful
SPK Short pass key
LPS Long pass successful
LPU Long pass unsuccessful
LPK Long pass key
TICS Tackle/interception/clearance successful
TICU Tackle/interception/clearance unsuccessful
TICK Key tackle/interception/clearance
SS Shot successful
SU Shot unsuccessful
SK Key shot
DPS Dribble/ prolonged possession successful
DPU Dribble/ prolonged possession unsuccessful
DPK Dribble/ prolonged possession key
HIS Header or interception with header successful
HIU Header or interception with head unsuccessful
HIK Key header or interception with head

13
Each successful performance indicator was assigned an „unsuccessful‟ and a „key‟ aspect
as a way of standardising the results (Hughes and Bartlett, 2002). A key aspect was
defined as the performance of an action which resulted in a key event, such as; a
successful pass leading to a shot on goal. Indicators were coded for use (table 1).

2.4. Recording and analysis procedure

One side of the pitch was demarcated at a distance of 10m beyond and parallel to the
sideline using coloured cones. The cones stretched 20m in width either side of the half
way line. The camera (Cannon MV 700i, 50Hz, Japan) was then fixed to a tripod
(Velbon, DV7000, Japan) within the coned area and pointed directly down the half way
line. Prior to recording, a supervisor from each team was asked to mark the number of
each player into a corresponding position on a pitch schematic. Elite players were
identified by a circle surrounding their number. Players were later identified by the pre-
determined code previously discussed.

The full match was played through and hand notated by the researcher, using the pre-
determined frequency model to note the player and associated behaviours (18
variables/indicators). The player was identified by shirt number, visual familiarization,
position on the field and by default (in comparison to other players).

2.5. Statistical Analyses

For the first and second stages of analysis, data were analyzed for normal distribution
using the Shapiro-Wilk statistic. Mann-Whitney U tests were subsequently used as the
most appropriate non-parametric determinant of statistical difference between two
independent groups (elite and sub-elite). Due to the high number of performance
indicators (18 dependent variables), a Bonferroni adjustment was applied reducing
statistical significance to p<0.0028, thus decreasing the risk of type 1 error (Vincent,
2005). A notable individual performance from a sub-elite player was demonstrated using
O‟Donoghue‟s (2005) method, using the normative percentile data of the position
matched elite population for comparison, thus forming stage 3 of analysis.

Inter-observer and intra-observer reliability was conducted using percentage error

calculations (Hughes et al., 2004, pg. 204) based on each of the 18 separate performance
indicators. Given that the analysis centred on performance indicators demonstrated by
players within respective positional groups, and that data were not collected sequentially,
it was deemed sufficient to analyse three full separate matches, randomly choosing one
positional group from each game (i.e. all defensive players, all midfield players and all
attacking players). For inter-observer reliability purposes, the lead researcher (+100 hours
on hand notation systems) and an experienced analyst (+100 hours on hand notation
systems) were chosen to analyse the three pre-selected groups in the same week, with the
aid of the definition dictionary to maintain objectivity. For intra-observer reliability, the
lead researcher re-analysed the same three positional groups following a one week
interval to account for memory effects. All statistical analyses and computations were
carried out on SPSS (version 13). Inter–observer reliability was conducted in the same
week.

14
From the 108 reliability calculations (6 x 18), 85 were within the 10% tolerance
suggested by Hughes et al. (2004). The remaining 23 variables ranged beyond the 10%
margin, 12 of which were „key‟ events. Although some of the results were calculated as
negligibly beyond the 10% tolerance, it appeared that the problematic indicators were
those which occurred less frequently (i.e. HIK, TICK and SPK). Cooper et al. (2007)
suggest how infrequent indicators may require more tolerable limits of error, since one
small absolute error may translate to a larger relative effect (i.e. as a percentage
difference). In contrast, the limited occurrence of some indicators, on occasion, seemed to
produce extremely low levels of error (see SS, SU and SK, DPK), possibly since key
events which occur less often may be easier to determine when based on a sound working
definition. In general, it was deemed that the majority of definitions assigned to each
indicator were appropriate for the respective positional groups in order to present the
example system. The limitations of key event indicators are discussed within the
following sections.

Table 2. Intra and inter-observer reliability for 3 positional groups each from 3 randomly
selected games.
Intra-Observer (% Diff) Inter-Observer (%Diff)

Performance Group Group

Indicator Defender Midfielder Attacker Defender Midfielder Attacker
SPS 3.2 4.5 1.0 5.3 2.2 2.2
SPU 0.0 2.2 7.4 6.4 3.3 0.0
SPK 0.0 0.0 10.5 40.0 22.2 10.5
LPS 4.5 8.7 11.7 6.9 8.0 25.0
LPU 0.0 7.4 9.5 6.0 7.4 8.7
LPK 7.4 0.0 0.0 6.9 8.7 18.1
TICS 6.0 7.4 9.5 0.0 0.0 10.5
TICU 8.0 6.8 0.0 8.0 0.0 0.0
TICK 9.5 0.0 18.1 11.7 7.4 40.0
SS 6.0 0.0 0.0 0.0 0.0 0.0
SU 0.0 8.7 0.0 15.3 8.7 0.0
SK 0.0 0.0 0.0 0.0 0.0 0.0
DPS 6.9 11.7 0.0 7.4 6.8 15.3
DPU 9.5 0.0 0.0 9.5 11.7 28.5
DPK 15.3 0.0 9.5 8.7 0.0 9.5
HIS 6.5 8.0 0.0 12.5 0.0 0.0
HIU 0.0 8.0 7.0 28.6 15.3 7.0
HIK 66.6 0.0 0.0 40.0 66.6 0.0
Note: Percentages expressed to one 1 decimal place, P1 = Player 1; P2 = Player 2. All
results over the 10% range are in bolded font.

15
3. Results

3.1. Overall testing of elite against sub-elite players

The results from a Mann-Whitney U collective comparison of elite against sub-elite
players indicated that externally pre-determined elite players demonstrated a significantly
higher (p<0.0028) performance level of game specific skill over 9 separate variables
(denoted by * in table 3). Of particular note, are the significantly higher variables of
successful shooting, successful passing and dribbling/prolonged possessions,
demonstrating significantly larger differences in three fundamental games specific skills.

Table 3. Overall five match comparison of elite and sub-elite players in all 18
performance variables

Performance Standard
Indicator
Sub-Elite Elite
(n=69) (n=32)
SPS 2.9±1.9 7.7±4.2*
SPU 2.6±1.6 2.4±1.7
SPK 0.1±0.3 0.8±0.9*
LPS 1.8±1.6 4.7±3.3*
LPU 2.3±1.7 2.5±2.1
LPK 0.1±0.5 0.7±1.1*
TICS 5±3.9 6±4.7
TICU 2±1.5 1.9±1.7
TICK 0.4±0.7 0.8±0.9
SS 0.3±0.6 1.2±0.9*
SU 0.1±0.3 0.6±0.8*
SK 0.2±0.4 0.6±0.6*
DPS 1.2±1.6 4.3±3.6*
DPU 1.3±1.3 1.6±1.4
DPK 0.1±0.5 1±1*
HIS 3.3±2.6 5.7±3.7
HIU 1.5±1.3 1.7±1.5
HIK 2±0.5 0.5±0.8
Note: Results presented as means and standard deviations.
Bonferroni adjusted alpha level (p< 0.0028). * = significant
difference (2 tailed).

3.2. Positional Grouping within competition levels

The results from the Mann-Whitney U testing, refined by positional differences and
competition level, demonstrated only one significant difference, with elite midfield
players performing more (p<0.0028) successful shots on goal during round 4 (see table
4). Although the reduced alpha level (as a result of the Bonferroni adjustment) may have

16
 masked the differences in some variables, the results suggest that differences between
ability groups are less apparent when refined by position.

Table 4. A comparison of elite and sub-elite players levels of game specific skill
performance inclusive of playing position and level of competition.
ROUND 4 ROUND 5

P.I. DEF MID ATT DEF MID ATT

SE E SE E SE E SE E SE E SE E
(n=20) (n=4) (n=15) (n=9) (n=8) (n=5) (n=13) (n=3) (n=9) (n=7) (n=4) (n=4)

SPS 2.5±1.6 4.5±1.9 4±2.2 9.4±4.6 2.1±1.6 8.8±4.7 3.0±2 2.6±2 3.3±1.5 7.2±2.7 2.7±1.7 10.3±3.
8
SPU 2.9±1.5 1±1.4 3±1.7 3.7±1 1.2±1.6 2.6±1.9 2.6±1.5 0.3±0.6 2.7±1.8 2.2±1.1 3.2±1.5 2.5±2.3
SPK 0.1±0.2 0.3±0.1 0.1±0.4 1.1±1 0.1±0.3 1±1 0.1±0.3 0.3±0.6 0.1±0.4 0.5±0.8 0.2±0.5 1.0±0.8
LPS 2.3±1.3 5.8±3.5 2.8±2.1 5.7±3.5 0.1±0.3 3.2±3.3 2.0±1.4 5.3±2 1.3±1.2 5.0±3.1 0.0±0 2.3±3.3
LPU 3.2±1.9 4.5±1.9 2.6±1.2 3.7±2.1 0.8±0.8 1.2±2.1 2.0±1.7 2.0±1 2.3±1.2 2.0±1.4 0.5±1 1.0±1.4
LPK 0.2±0.4 0.3±0.5 0.3±1 1.5±1.6 0±0 0.4±0.5 0.1±0.3 0.6±1.2 0.1±0.3 0.4±0.8 0.0±0 0.2±0.5
TICS 7±4.1 13±3.5 5±2.1 4.7±3.8 0.5±0.5 3.6±1.8 6.6±5.1 12.6±2 3.5±1.1 4.0±2.7 3.2±2.8 3.5±3.3
TICU 2.6±1.8 1.3±0.9 2.1±0.9 1.9±1.6 1.3±1.4 2±2.9 1.8±1.9 0.3±0.6 1.7±0.9 3.2±1.1 1.7±1.7 1.5±1
TICK 0.6±0.8 1.5±1.3 0.3±0.4 0.3±0.5 0±0 0.4±0.5 0.3±1.2 2.6±0.6 0.2±0.4 0.5±0.8 0±0 0.2±0.3
SS 0.1±0.3 0.3±0.5 0.4±0.5 1.4±0.5 0.8±1.1 1.2±1.7 0.1±0.4 0±0 0.2±0.4 1.2±0.9 0.7±0.9 2.0±0.8
*
SU 0.1±2.2 0±0 0.2±0.5 0.6±0.8 0.1±0.4 1.4±0.9 0.1±0.3 1±1 0±0 0.1±0.3 0.2±0.5 0.7±0.5
SK 0.1±2.2 0±0 0.2±0.4 0.6±0.5 0.4±0.7 0.6±0.9 0±0 0±0 0.2±0.4 0.5±0.5 0.5±0.6 1.5±0.6
DPS 0.9±1.2 0.5±1 1.9±2.3 6.3±4.3 1.1±1.3 3.2±1.6 0.9±1 1±1 1.8±1.5 5.2±2.9 0.5±1 6.0±4
DPU 1.2±1.2 0.5±0.6 0.7±0.7 2.3±1.8 1.8±1.2 2.6±1.4 0.8±1.1 0±0 2.4±1.6 1.4±0.8 2.2±0.9 1.2±0.5
DPK 0.1±0.2 0±0 0.3±1 1.3±1.2 0.1±0.4 0.4±0.5 0.1±0.3 0.6±0.5 0.1±0.3 1.1±1.2 0±0 1.7±0.9
HIS 3.2±1.9 9±1.8 2.9±2.1 4.7±3.8 1.3±1.6 7.4±3.7 5.3±3.9 9.6±2 3.5±1.8 2.5±1.6 2.5±2.3 5.0±3.9
HIU 1.8±1.4 1.5±1.3 1.5±1.3 1.3±1 1.1±1.7 2.8±2.4 1.6±1.2 1.0±1 1.2±1.2 1.8±1.3 0.7±1.5 1.2±1.5
HIK 0.2±0.5 1±1.4 0.1±0.3 0.1±0.3 0.1±0.3 1±0.7 0.4±0.8 1.6±0.6 0.1±0.3 0±0 0±0 0.7±0.9
Note: Results presented as means and standard deviations. Bonferroni adjusted alpha level
(p< 0.0028). * = significant difference (2 tailed). DEF= Defender, MID= Midfielder, ATT=
Attacker. SE= Sub-Elite. E= Elite. P.I. = Performance Indicator

3.3. Individual Profiling

In order to demonstrate the possible uses of O‟Donoghue‟s (2005) method for normative
profiling, sub-elite player A31/B15 from school 5 was selected based on the levels of
performance shown throughout rounds 4 and 5 of the competition. Figure 2 presents the
performance of player A31/B15 relative to the percentile range of elite performers
throughout rounds 4 and 5. Attention should be drawn towards HIK, where the player
produced levels at the 100% range. Furthermore, promising results were demonstrated in
the TICS performance indicator (60%) and the HIS category (70%). It is also interesting
to observe the 70% recorded performance of the HIU, which appears to demonstrate the
variable performances of this indicator by player A31/B15, denoted by the large inter-
quartile range (40% to 80%).

17
Figure 2. Normative Performance data of sub-elite midfield player against the elite
positional group (n=8) over 2 rounds of competition. Note: For any variables concerning
unsuccessful indications i.e. LPU. The objective was to reach a lower score. Missing
values i.e. unsuccessful shots on goal (SU) show where the player did not produce any of
these actions.

4. Discussion

Results from stage 1 of analysis indicated that nine of the eighteen variables (50%) were
significantly higher (p<0.0028) for all elite performers compared to all sub-elite
performers. On this basis, the first research objective was met, in that the variables
outlined in table 3 may provide a feasible starting point for potential indicators that
generally distinguish between elite and sub-elite players of an Under 14‟s age group
whilst competing within a shared extracurricular context. The areas of particular note are
the significantly higher performances of both long and short passing and the key
contributions for each. Both successful dribbling/prolonged possessions and key
dribbling/prolonged possessions were also higher in the elite group as a whole,
conforming to the results from the closed skill testing of previous research, where
dribbling was a constant determinant of elite players (Huijgen et al., 2009; Vaeyens et al.,
2006; Reilly et al., 2000b). Caution is also warranted when comparing previous research

18
to the current study, given the methodological disparity, namely the use of closed or open
skill testing. Nonetheless, the results provide an argument that fine motor skills in an
open or closed environment, may strongly characterise elite players, informing
researchers and practitioners of suitable skill criteria (Reilly et al., 2000b) to be used in
future identification procedures if, indeed, this is the approach that research intends to
adopt.

There were, however, some contrasts to the work of Reilly et al. (2000b), where sub-elite
players were reportedly more proficient in shooting performance, with results from table
3 showing that successful and key shooting performance was significantly higher in elite
compared to sub-elite players. However, whilst successful shooting performance
indicators were significantly higher in elite players, the absolute frequency remained
remarkably low (i.e. successful shooting; 1.2±0.9), demonstrating that elite youth players
may often demonstrate higher performance than sub-elite counterparts, yet have limited
opportunities to perform the required skills. Such measures may, therefore, be the direct
result of a higher efficiency within other skills. For example, the ability to dribble the ball
may forge the opportunity to shoot on goal. Interestingly, unsuccessful shooting was also
statistically higher in elite players, demonstrating that elite players attempt shots on goal
more often, even if the outcome is unsuccessful. Such results firstly suggest that skills
should not be considered in isolation whilst attempting to differentiate between ability
groups, rather used in conjunction with other skilled behaviours to create an overall
impression of performance. Secondly, the current results may also suggest a contradiction
between using closed and open skill testing to measure game specific skill, since a sub-
elite player may merely be able to shoot on goal accurately during a closed skill test yet
experience difficulty in performing this skill under less predictable conditions. Therefore,
the results indicate that the lack of ecologically valid protocols used previously, may not
test for skill accurately according to academic definitions (see Sunderland et al., 2006),
nor provide a value that represents the potential interaction of game specific skills during
open match conditions.

Defenders, midfielders and attackers were further independently tested within identical
levels of competition in an attempt to reduce the effect that higher competitive levels and
specific positioning may have on game specific skill output. The variable of successful
shooting in elite midfielders (1.4±0.5 to 0.4±0.5) of round 4 proved to be the only
significantly higher variable (p<0.0028). Coupled with the results from the overall
comparisons, it is clear that, alongside the impact of a closed environment on game
specific performance, both the influence of competition level and the position of a player
may have a substantial impact on results. As a result of the current research, it may be
more difficult than previously thought to distinguish between elite and sub-elite players
based on positional groups, posing a problem for future research, especially if the desire
is to identify players based on positional preferences. However, it is more likely, in this
case, that certain issues with the current prototypal methodology have limited the extent
of the results during this refined stage of testing.

An inherent issue within the current system appeared to lie in the amount of variables
used (eighteen in total), which caused subsequent problems distinguishing differences

19
whilst using Mann-Whitney U testing, since the Bonferroni adjustments reduced the
alpha level to p<0.0028. As a result, the sensitivity to differences in positional groups
was decreased, even under somewhat disparate results. For example, the comparison of
DPS in elite (5.2±2.9) and sub-elite (1.8±1.5) midfielders in round 5 demonstrated a
potentially high level of significance (p = 0.010), which under less intolerable
circumstances may have been enough to denote a statistical difference. The degree of
variability demonstrated in each performance indicator, represented by the associated
standard deviations, may have also masked the potentially larger differences in
performance indicators between ability groups. Such variable differences between
individuals and inconsistencies within individuals are, however, an inevitable feature of
youth performance.

Future attempts to emulate the present system may, consequently, consider excluding
„key‟ indicators to account for the aforementioned issue. Key indicators appear
appropriate for future exclusion due to the additional issues with reliability outlined in
table 2, causing difficulties in making accurate inferences based on these indicators. With
regards to inter-observer reliability, clear work has to be done in order to improve the
present system for future application with different users. Reliability issues do not appear
to be related to definitions, rather to the performance indicator itself. For example, the
key event indicators (SK, TICK, HIK) were commonly differing by merely one notation
but, due to the low level of occurrence in absolute terms, this often translated into a larger
(>10%) percentage difference. It is acknowledged that data could have been used in a less
refined format (i.e. a percentage difference of all HIK by every player from every round).
However, using larger summary groups of comparisons may produce misleading results
and should be left at the same level of eventual data processing (Hughes et al., 2004). As
a result, future research may seek to source a larger pool of data with which to perform
comparisons, thus reducing the impact of less frequently occurring performance
indicators.

Results from the chosen player (A31/B15) derived from the normative profiling analysis,
demonstrated clear strengths and weaknesses in the relative performance of this „sub-
elite‟ player compared to elite, position matched counterparts. It should be remembered
that the chosen players performance is particularly noteworthy, given the fact that figure
2 reflects a comparison of one sub-elite performer against a percentile range of all the
elite midfield players from rounds 4 and 5. Interestingly, however, the performance
strengths of the player in question demonstrated no correspondence to that of stage 1 of
analysis, with no notable congruence to the variables highlighted as significant to elite
performance (table 2). Therefore, the current player appears to be performing well in
areas that are less associated with elite performance, which may provide one reason as to
why the player has previously gone unnoticed through traditional scouting practices. As
suggested by Vaeyens et al. (2008), basing such a profiling method within a suitable
developmental environment, may provide a valuable tool to monitor trends in the relative
performance of a player‟s game specific skill alongside other objective measures (i.e.
physiological or psychological) to inform a broader talent identification process.

20
It is important to understand the relative performance of individuals since the demands of
each match played may be variable, preventing the obtainment of stable criterion values.
To highlight the potential uses of the current profiling method further, it would be
possible to perform a comparison of the relative performance of any group of chosen
players against position matched counterparts. As such, a longitudinal design may be
applied in order to record an appropriate depth of data with which to perform such a
comparison. It is clear that, when used in the correct manner, such information could be
invaluable for talent identification systems as an alternative to values derived from closed
skill testing. The current system could be easily adopted and adapted as part of the multi-
dimensional talent model, providing a longitudinal, ecologically valid and objective
measure of game specific skills which is currently emerging as a preferred approach to
talent identification (Vaeyens et al., 2008).

The use of normative profiling also provides a more flexible framework, escaping the
rigid „criterion‟ methods that are suggested in previous research (Reilly et al., 2000b).
Using the current method, a player can be judged based on weighted criteria, which, for
example, could draw upon a percentile range of elite players over the last ten games for
comparative purposes, thus accounting for current form. Likewise, a larger database,
spanning previous years of player analysis could be used to trend levels of individual
performance. The possibility to track the trainability of players (Gabbett et al., 2007;
McCrory, 2005; Van Rossum and Wijbenga, 1993), determine areas for player
improvement and recognise the antecedents of current successful players may
subsequently become a reality. The associated spread of performance against a chosen
global data set that accompanies O‟Donoghue‟s (2005) method also provides an
opportunity to identify players who exhibit changes in sport specific motor skill during
later stages of maturation (Malina et al., 2005) or perform inconsistently (demonstrated
by selected variables in figure 2), thus noting the „potential‟ of a given player to be either
better or worse than an average value may suggest. Furthermore, the value of
understanding the nature of skill variability over maturation may provide support to
currently advocated talent identification and development models (see Vaeyens et al.,
2008), embracing the inevitable changes in performance that accompany the learning and
growing process (Pearson et al., 2006).

5. Conclusion

Talent has been described as the recognition of potential by means of certain measures,
accepted as markers of future high performance (Howe et al., 1998). It is the lack of
agreement on „certain accepted measures‟ and whether these demonstrate any
significance towards future performance, which remains the standing problem. It has
been previously suggested that objective measurement may not identify potential as it
merely reflects current performance, which is variable throughout childhood and
adolescence (Abbot and Collins, 2002). However, using O‟Donoghue‟s (2005) method,
the current system has demonstrated the potential to account for long term variability in
performance of game specific skills and how this could be further developed to inform a
broader talent identification process. It is acknowledged that potential indications of elite

21
performance may be demonstrated in many other ways, as per the multi-dimensional
approach, however, the clear advantages of ecologically valid measurements to the
assessment of game specific skills has been highlighted. The continued integration of
„real world‟ approaches in future research may increase the validity of measures designed
to recognise markers of potential sporting success.

6. References

Abbott, A. and Collins, D. (2002). A theoretical and empirical analysis of a „state of the
art‟ talent identification model, High Ability Studies, 13, 157-178.
Abbott, A., Button, C., Pepping, G. J. and Collins, D. (2005). Unnatural selection: talent
identification and development in sport. Psychological Life Science, 9, 61-88.
Cooper, S. M., Hughes, M., O'Donoghue, P. and Nevill, A. M. (2007). A simple
statistical method for assessing the reliability of data entered into sport performance
analysis systems. International Journal of Performance Analysis in Sport, 7,
(1), 87-109.
Elferink-Gemser, M., Visscher, C., Lemmink, K. and Mulder, T. (2004). Relation
between multi-dimensional performance characteristics and level of performance in
talented youth field hockey players. Journal of Sport Sciences, 22, 1053-1063.
Falk, B., Lidor, R., Lander, Y. and Lang, B. (2004). Talent identification and early
development of elite water-polo players: a 2-year follow-up study, Journal of
Sport Sciences, 22, 347-355.
Gabbett, T., Georgieff, B. and Domrow, N. (2007). The use of physiological,
anthropometric and skill data to predict selection in a talent identified junior
volleyball squad. Journal of Sport Sciences, 25, 1337-1344.
Hammond, J. (2001). Talent identification and development in Indian football. Soccer
and Society, 2, 146-161.
Howe, M. J. W., Davidson, J. W. and Sloboda, J. S. (1998). Innate Talents: Reality or
Myth. Behavioural and Brain Sciences, 21, 399-442.
Hughes, M. and Bartlett, R. M (2002). The use of performance indicators in performance
analysis, Journal of Sport Sciences, 20, 739-754.
Hughes, M., Evans, S. and Wells, J. (2001). Establishing normative profiles in
performance analysis, International Journal for Performance Analysis in Sport,
1, 4–27.
Hughes, M., Evans, S. and Wells, J. (2004). Establishing normative profiles in
performance analysis. In M. D, Hughes and I. M., Franks (Eds). Notational
Analysis of Sport: systems for better coaching and performance in sport (2nd
Edition) (pp. 205-226). London; New York: E and FN SPON.
Hughes, M. D., Cooper, S. M. and Nevill, A. M. (2004). Analysis of notation data. In M.
D, Hughes and I. M., Franks (Eds). Notational Analysis of Sport: systems for
better coaching and performance in sport (2nd Edition) (pp. 189-204). London;
New York: E and FN SPON.
Huijgen, B. C. H., Elferink-Gemser, M. T., Post, W. J. and Visscher, C. (2009). Soccer
skill development in professionals. International Journal of Sports Medicine, 30,
585– 591.

22
James, N. (2006). Notational Analysis in Soccer: past, present and future. International
Journal of Performance Analysis in Sport, 6, (2), 67-81.
James, N., Mellalieu, S. D. and Hollely, C. (2002). Analysis of strategies in soccer as a
function of European and domestic competition. International Journal of
Performance Analysis in Sport, 2, (1), 85-103.
James, N, Mellalieu, S. and Jones, N. (2005). The development of position-specific
performance indicators in professional rugby union. Journal of Sports Sciences,
23, (1), 63-72.
Kozel, J. (1996). Talent Identification and Development in Germany. Coaching focus,
31, 12-13.
Malina, R. M.., Cumming, S. P., Kontos, A. P., Eisenmann, J. C., Ribero, B. and Aroso,
J. (2005). Maturity-associated variation in sport-specific skills of youth soccer
players aged 13 – 15 years. Journal of Sports Sciences, 23, (5), 515–522.
Martindale, R. J. J., Collins, D. and Abraham, A. (2007). Effective talent development:
The elite coach perspective in UK sport. Journal of Applied Sport Psychology,
19, 187-206.
McCrory, P. (2005). At the sharp end. British Journal of Sports Medicine, 39, 1.
O'Donoghue, P. G. (2005). Normative profiles of sports performance. International
Journal of Performance Analysis of Sport, 5, (1), 104-119.
Pearson, D. T., Naughton, G. A. and Torode, M. (2006). Predictability of physiological
testing and the role of maturation in talent identification for adolescent team sports.
Journal of Science and Medicine in Sport, 9, 277-287
Pienaar, A. E., Spamer, M. J. and Steyn, H. S. (1998). Identifying and developing rugby
talent among ten year old boys: a practical model, Journal of Sport Sciences, 16,
691-699.
Reilly, T. and Holmes, M. (1983). A preliminary analysis of selected soccer skills,
Physical Education Review, 6, 64-71.
Reilly, T., Bangsbo, J. and Franks, A. (2000a). Anthropometric and physiological pre-
dispositions for elite soccer, Journal of Sport Sciences, 18, 669-683.
Reilly, T., Williams, A. M., Nevill, A. and Franks, A. (2000b). A multidisciplinary
approach to talent identification in soccer. Journal of Sports Sciences, 18, 695-
702.
Rowlinson, M. and O‟Donoghue, P. (2008). Performance profiles of soccer players in the
2006 UEFA Champions League and the 2006 FIFA World Cup tournaments. In T.
Reilly and F. Korkusuz (Eds.), Science and Football VI, (pp.229-234). London:
Routledge.
Sunderland, C., Cooke, K., Milne, H. and Nevill, M. E. (2006). The reliability and
validity of a filed hockey skill test. International Journal of Sport Medicine, 27,
395-400.
Tenga, A. and Larsen, Ø. (2003). Testing the validity of match analysis to describe
playing styles in football. International Journal of Performance Analysis in
Sport, 3, 90-102.
Vaeyens, R, Lenoir, M., Williams, M. A. and Philippaerts, R. M. (2008). Talent
identification and development programmes in sport current models and future
directions. Sports Medicine, 38, (9), 703-714.

23
Vaeyens, R., Malina, R. M., Janssens, M., Van Renterghem, B., Borgois, J., Vrijens, J.
and Philippaerts, R. M. (2006). A Multidisciplinary selection model for youth
soccer: the Ghent Youth Soccer Project. British Journal of Sports Medicine, 40,
928-934.
Van Rossum, J. H. A., and Wijbenga, D. (1993). Soccer skills technique tests for youth
players: construction and implications. In T. Reilly, J. Clarys and A. Stibbe (Eds.),
Science and Football II (pp. 313 -318), London: E and FN Spon.
Vincent, W. J. (2005). Statistics in Kinesiology. Champaign, IL: Human Kinetics.
Williams, A. M. and Reilly, T. (2000). Talent identification and development, Journal of
Sport Sciences, 18, 657-667.

24