Influence of DEM Resolution On Drainage Network Extraction A PDF

You might also like

Download as pdf or txt
Download as pdf or txt
You are on page 1of 12

Geomorphology 241 (2015) 243254

Contents lists available at ScienceDirect

Geomorphology
journal homepage: www.elsevier.com/locate/geomorph

Inuence of DEM resolution on drainage network extraction: A


multifractal analysis
A.B. Ariza-Villaverde , F.J. Jimnez-Hornero, E. Gutirrez de Rav
Department of Graphic Engineering and Geomatic, University of Cordoba, Gregor Mendel Building (3rd oor), Campus Rabanales, 14071 Crdoba, Spain

a r t i c l e

i n f o

Article history:
Received 11 August 2014
Received in revised form 23 March 2015
Accepted 27 March 2015
Available online 18 April 2015
Keywords:
Drainage network geometry
DEM resolution
D8 algorithm
Multifractal analysis
Flow accumulation threshold
Terrain attributes

a b s t r a c t
Different hydrological algorithms have been developed to automatically extract drainage networks from digital
elevation models (DEMs). D8 is the most widely used algorithm to delineate drainage networks and catchments
from a DEM. This algorithm has certain advantages such as simplicity, the provision of a reasonable representation for convergent ow conditions and consistency among ow patterns, calculated contributing areas and the
spatial representation of subcatchments. However, it has limitations in selecting suitable ow accumulation
threshold values to determine the pixels that belong to drainage networks. Although the effects of DEM resolution on some terrain attributes, stream characterisation and watershed delineation have been studied, analyses of
the inuence of DEM resolution on ow accumulation threshold values have been limited. Recently, multifractal
analyses have been successfully used to nd appropriate ow accumulation threshold values. The application of
this type of analysis to evaluate the relationship between DEM resolution and ow accumulation threshold value
needs to be explored. Therefore, this study tested three DEM resolutions for four drainage basins with different
levels of drainage network distribution by comparing the Rnyi spectra of the drainage networks that were obtained with the D8 algorithm against those determined by photogrammetric restitution. According to the results,
DEM resolution inuences the selected ow accumulation threshold value and the simulated network morphology. The suitable ow accumulation threshold value increases as the DEM resolution increases and shows greater
variability for basins with lower drainage densities. The links between DEM resolution and terrain attributes
were also examined.
2015 Elsevier B.V. All rights reserved.

1. Introduction
A digital elevation model (DEM) is a numerical representation of the
Earth's surface that contains the terrain height (Felicsimo, 1994). A
DEM can be dened by means of i) contours with x, y coordinate pairs
along each contour line of a specied elevation, ii) a triangulated irregular network made up of irregularly distributed nodes and lines with
three-dimensional coordinates (x, y, and z) and iii) a two-dimensional
array of numbers that represents the spatial distribution of elevations
on a regular grid. The main advantage of the third denition is its computational efciency and lower storage requirements (Walker and
Willgoose, 1999). Thus, raster DEMs have been broadly applied to
modelling certain features of the surface hydrology such as catchments
(O'Callaghan and Mark, 1984; Jones, 2002), drainage networks
(O'Callaghan and Mark, 1984; Turcotte et al., 2001), channel heads
(Montgomery and Dietrich, 1988; Julian et al., 2012) and wetness

Corresponding author. Tel.: +34 957 212126; fax: +34 957 218455.
E-mail addresses: g82arvia@uco.es (A.B. Ariza-Villaverde), fjhornero@uco.es
(F.J. Jimnez-Hornero), eduardo@uco.es (E. Gutirrez de Rav).
URL: http://orcid.org/0000-0003-4498-8797 (F.J. Jimnez-Hornero).

http://dx.doi.org/10.1016/j.geomorph.2015.03.040
0169-555X/ 2015 Elsevier B.V. All rights reserved.

indices (Beven and Kirkby, 1979; Vaze et al, 2010). A DEM reproduces
real hydrological features with accuracy and computational efciency
that are determined by its resolution, vertical and horizontal precision
and terrain attributes (Moore and Grayson, 1991; Quinn et al., 1991;
Wolock and Price, 1994; Deng et al., 2007; Wu et al., 2008a, 2008b;
Dehvari and Heck, 2013).
Several studies have veried the link between simulated hydrological features and DEM resolution. Zhang and Montgomery (1994)
explored the effects of grid size on landscape representation and hydrologic simulations. The lower the DEM resolution (i.e., large grid size), the
higher the mean topographic index because of increasing contributing
area and lower slopes. Similarly, Srensen and Seibert (2007) studied
the effects of DEM resolution on the calculation of topographic wetness
and established notable differences between different grid resolutions.
Kenward et al. (2000) analysed the effects of the vertical precision of
DEMs on the accuracy of hydrology predictions, pointing to the reduced
spatial coherence in images with lower vertical precision, and concluded that topography plays a fundamental role in hydrologic and geomorphologic modelling. This statement has been conrmed by Falorni et al.
(2005) based on the strong inuence of relief on the vertical accuracy of
DEMs and derived terrain attributes. Regarding the effects on DEM resolutions, McMaster (2002) explored the effects of DEM resolution on

244

A.B. Ariza-Villaverde et al. / Geomorphology 241 (2015) 243254

the positional accuracy of derived hydrologic networks by using two


hydrological algorithms: D8 (O'Callaghan and Mark, 1984) and D8LAT (Tarboton, 1997). Chaubey et al. (2005) also described the inuence of DEM resolution on watersheds, stream network delineation
and sub-basin classication by using the SWAT model (Soil and Water
Assessment Tool), which is a river basin scale model to quantify the
impact of land management practices in large, complex watersheds. Accordingly, the accuracy of the simulation of hydrological processes also
depends on the algorithms that are used to extract hydrologic features
from DEMs. The rst approach developed to extract river networks
was the D8 algorithm, which uses a neighbourhood of eight cells as possible ow directions. However, the results of applying this method are
sometimes non-realistic (Turcotte et al., 2001) due to the determination
of the ow in only one of the eight possible directions, the presence of
at areas and pits and the lack of information on lake locations. The multiple ow direction method (MFD) was suggested by Freeman (1991) to
overcome the limitations of D8. This approach does improve the D8
model in some aspects, but it requires additional computational time
to calculate a greater density of ow connections (Gallant and Wilson,
1996), and the ow from a pixel is dispersed to all of the neighbouring
pixels that have lower elevations. Lea (1992) developed an algorithm
that calculates the ow direction through the aspect associated with
each cell. It has the advantage of specifying ow direction continuously
and without dispersion. Costa-Cabral and Burges (1994) developed the
DEMON (Digital Elevation Model Network) algorithm to improve the
modelling proposed by Lea (1992). Both plane ow methods are deterministic and resolve ow directions. However, as Tarboton (1997) stated, they are susceptible to problems that arise from the approximation
involved in tting a plane through four points. Tarboton (1997) and
Orlandini et al. (2003) have proposed the D8-LAD and the D8-LTD
methods, respectively, to overcome some of the limitations of the D8 algorithm, such as the effects of grid artefacts (i.e., pits and ats areas),
high dispersion and signicant computational costs. However, both
methods have shortcomings. The removal of the unimodal link between
ow directions and the difculty in calculating the catchment boundary
due to the multiple ow directions from a cell (Orlandini et al., 2003)
are the main drawbacks of the D8-LAD algorithm. On the other hand,
users have to specify the dampening factor for the D8-LTD model. To improve this algorithm, Paik (2008) proposed a global search method, GD8,
and argued that there is no analytical justication to x the dampening
factor to 1, as Orlandini et al. (2003) recommended. Orlandini and
Moretti (2009) replied to Paik (2008) and afrmed that the analytical
background does exist and is easily illustrated by considering a portion
of a sloping plane. Later, Orlandini et al. (2014) demonstrated the validity
of their method to determine slope lines in grid digital elevation models,
which is the key factor for obtaining the smallest transverse deviations.
As described above, recent alternatives to the D8 method exhibit
drawbacks that limit their application. As a consequence, the D8 algorithm is the most commonly used (Tribe, 1992; Martz and Garbrecht,
1998; Saunders, 1999; Jones, 2002). Its popularity stems from its simplicity and reasonable representation for convergent ow conditions
(Freeman, 1991), and it preserves the consistency between the ow
patterns, calculated contributing area and spatial delineation of subcatchments (Martz and Garbrecht, 1992). The D8 method is widely
used in GIS packages, for example ESRI's ArcGIS software, and is
frequently used to study different hydrological processes (Cai et al.,
2012; Gericke and Venohr, 2012; Babbar-Sebens et al., 2013; LpezVicente et al., 2013; Rodriguez et al., 2013).
The ow accumulation threshold value is a parameter of the D8
algorithm whose suitable determination allows the selection of
DEM cells that represent drainage networks. Tarboton et al. (1988),
Montgomery and Dietrich (1992) and Orlandini et al. (2011) have suggested that the threshold value may strongly depend on DEM resolution. The quality of generated drainage networks is inuenced by DEM
resolution (McMaster, 2002; Chaubey et al., 2005) because it affects
the basin slope and, as a consequence, the upslope contributing area

(Thieken et al., 1999; Walker and Willgoose, 1999; McMaster, 2002;


Wu et al., 2008a, 2008b). Thus, the ow accumulation threshold value
needs to be modied according to the resolution of the DEM.
Ariza-Villaverde et al. (2013) determined the appropriate threshold
value to a high accuracy by means of a multifractal analysis for a DEM
resolution of 10 m. Different multifractal methods have been successfully applied to the study of river network morphology (Rinaldo et al.,
1993; De Bartolo et al., 2000, 2004, 2006a, b) and to analyse different
variables such as the inuence of lithological and tectonic morphologies
on rivers (Gaudio et al., 2006; Dombrdi et al., 2007). The multifractal
theory implies that the complex and heterogeneous behaviour of a
self-similar measure (i.e., statistically similar on any scale) can be represented as a combination of interwoven fractal sets with corresponding
scaling exponents. The advantages of the multifractal approach are
that its parameters are independent over a range of scales and that no
assumption is required about the data following any specic distribution. Two main classes of multifractal algorithm can be distinguished:
xed-size algorithms (FSA) and xed-mass algorithms (FMA), which
were introduced by Badii and Politi (1984a, b, 1985). The rst class is
suitable for multifractal objects in which the original region is initially
divided into several pieces; each piece is sub-divided at each step into
other pieces, the size of which is reduced by a constant factor. The
approaches include the box-counting method (Russel et al., 1980), the
sandbox method (Tl et al., 1989; Vicsek, 1990; Vicsek et al., 1990)
and the generalised correlation integral method (Pawelzik and
Schuster, 1987), which have been applied to the study of river networks
(Rinaldo et al., 1992, 1993; Rigon et al., 1993; De Bartolo et al., 1995,
2000, 2004, 2006a). The FMA algorithms are appropriate if the measure
of each piece is reduced by a constant factor (Mach et al., 1995). In this
method, the quantity that is held xed is no longer the size of the covering boxes, but the measure inside the box. De Bartolo et al. (2006b) applied the xed-mass algorithm to the analysis of river networks and
braided channels with very good results. In general, FSA are advantageous for their computational aspects (Gaudio et al., 2004). However,
some of these algorithms pose challenges with regard to the right side
of the spectrum, i.e., for negative moment orders, and also for the analysis of border effects when networks are analysed by emphasising regions that have few data points (De Bartolo et al, 2000). To solve these
problems, the sandbox method and the generalised correlation integral
method have proven to be able to reconstruct the complete multifractal
spectrum (left and right sides) and to solve the border effects, whereas
the box-counting method fails in the case of negative moment orders.
Multifractal parameters known as information, capacity and correlation dimensions have been suggested as suitable to describe drainage network morphology (Saa et al., 2007; Ariza-Villaverde et al., 2013). Thus,
the inuence of DEM resolution on the D8 algorithm threshold ow accumulation value is explored here by using the Sandbox multifractal method. The accuracy of the obtained drainage networks will be checked by
comparing them against the accuracy of photogrammetric restitution.
2. Study area
Four river basins located in the north of the province of Cordoba,
Andalusia, southern Spain, were selected for this study (Fig. 1). The basins have different degrees of drainage density. The basins' locations can
be found on sheet number 922 of the 1:50,000 territorial division national maps published by the National Geographic Institute of Spain.
The corresponding DEMs with 5, 10 and 25 m resolutions were obtained
from photogrammetric ights in 2010 and generated by automatic correlation (scale 1:25.000). In addition, the drainage networks in the
study area from photogrammetric restitution provided by the Department of Agriculture, Fisheries and Environment, Government of
Andalusia, were used to check the results from the D8 algorithm.
The study basins present maximum and minimum heights of 154
and 584 m, respectively, and an average slope of 30%. The main types
of soil that are prevalent in the study basins are regosol, lithosol and

A.B. Ariza-Villaverde et al. / Geomorphology 241 (2015) 243254

245

Fig. 1. Locations and DEMs of the study basins.

cambisol, with rankers on metamorphic rocks (slates) according to the


Soil Map of Andalusia (Department of Agriculture, Fisheries and Environment, Andalusian Autonomous Government), and the vegetation
classication corresponds to Luso-Extremadurense according to the
Map Vegetation Series of Spain (Rivas-Martnez, 1987), with cork oak
(Quercus suber) and holm oak (Quercus ilex) forests.
3. Methodology
3.1. Hydrological model in GIS: ArcHydro
ArcHydro is an extension of the water resource applications developed for ArcGIS Desktop by the University of Texas (Maidment, 2002).
Among other hydrological applications, this tool can be used to

efciently delineate watersheds and generate stream networks by


means of the D8 algorithm. An initial DEM that contains the terrain
height in each cell or pixel is used as an input to identify stream channels. Flow directions are calculated taking into account the corrections
of the DEM's inaccuracies and anomalies. After that, a ow direction is
assigned to each pixel according to Fig. 2, depending on the maximum
slope of the cells around it. A ow accumulation map is obtained in
the next step, which allows the denition of river networks. Cells
whose ow accumulations are higher than or equal to the selected
ow accumulation threshold value will comprise the drainage network.
Maidment (2002) suggests considering a threshold value that is equal
to 1% of the maximum ow accumulation, although any other value
can be selected. In this research, the D8 algorithm was applied for
threshold ow accumulation values that ranged from 10 to 900.

Fig. 2. Flow routing scheme of the D8 method.

246

A.B. Ariza-Villaverde et al. / Geomorphology 241 (2015) 243254

According to Ariza-Villaverde et al. (2013), threshold values equal to or


greater than 1% of the maximum ow accumulation yield drainage networks sparser than real networks.
3.2. Multifractal analysis: sandbox method
The sandbox method was initially proposed by Tl et al. (1989)
and developed by Vicsek (1990) and Vicsek et al. (1990). First, the
method randomly selects N points that belong to the drainage network.
For each point i, the algorithm counts the number of pixels Mi(R) that
belong to the drainage network inside a region of a given radius R
(i.e., a circle in 2D) centred at this point.
Choosing arbitrary network points as centres, the average value of
the mass and its q-th moments over randomly distributed centres can
be computed as [M(R)]q, where q is the probability moment order.
Thus,
X
i

Mi
M0

q1

 q1Dq
Mi
R

;
L
M0

where M0 is the total mass of the cluster or lattice mass, and L is the lattice size, which is equal to 1 after normalisation. This normalisation does
not modify the measure, as it is a geometrically invariant transformation
(Falconer, 1990).
Considering the ratio Mi/M0 as a probability distribution on an approximating fractal, the following averaged expression can be derived
as follows:
*
 +  q1Dq
M R q1
R
:

M0
L

According to Eq. (2), the selection of the centres must be uniform on


the approximating fractal.
The generalised fractal dimension, Dq, of moment order q is dened
as (Tl et al., 1989)
D
E
ln M R=M0 q1
1
lim
for q 1:
Dq R=L
q1 R=L0
ln R=L

De Bartolo et al. (2004) obtained the singularity for q = 1 through a


Taylor series expansion of approximately 1 + dq:
Dq R=L lim

R=L0

ln hMR=M 0 i
:
ln R=L

The generalised dimensions can be obtained through a least squares


linear regression as the slope of the scaling curves ln [M(R)/M0]q 1
versus ln(R/L) q 1 and ln[M(R)/M0] versus ln(R/L) for q = 1, between ln(R/L)lower and ln(R/L)upper, which are the lower and upper
cut-off lengths.
The maximum and minimum normalised radii R/L chosen for calculating the algorithm are between 0.0028 and 0.25 for basin 1, 0.0047
and 0.25 for basin 2, 0.0043 and 0.25 for basin 3 and 0.0038 and 0.25
for basin 4. The minimum radius is selected so that two pixels or two
drainage network points must be inside the circle, and the maximum
cannot be greater than 25% of the lattice size.
The fractal dimensions Dq that provide information about the
drainage network morphology occur for q values that are equal to 0, 1
and 2. D0 is the fractal dimension of the set over which the measure is
carried out, and it refers to the degree of lling space by drainage networks. D1 is the information dimension that describes the degree of heterogeneity in the distribution of the measure. According to Davis et al.
(1994), the information dimension D1 provides a measure of the degree
of heterogeneity in the spatial distribution of a variable. In addition, D1
characterises the distribution and intensity of singularities with respect

to the mean. If D1 becomes smaller, the distribution of the singularities


in the drainage network density will be sparse. On the contrary, if D1
increases, the singularities will have lower values that exhibit a more
uniform distribution. D2 is the correlation fractal dimension, which is associated with the correlation function, and it determines the average
distribution of the measure (Grassberger, 1983; Grassberger and
Procaccia, 1983). It describes the uniformity of the drainage network
density among several selected zones. It describes the probability of
nding pixels that belong to the object within a given distance when
starting on a pixel that belongs to the object. Dq is a decreasing function
with respect to q for a multifractally distributed measure (e.g., Saa et al.,
2007) where D0 N D1 N D2.
4. Results and discussion
4.1. Multifractal analysis of the ow accumulation threshold value obtained
for different DEM resolutions
Fig. 3 shows the drainage networks that were obtained by the D8
algorithm for 1% of the maximum ow accumulation value and the
drainage networks determined by photogrammetric restitution. The
drainage networks derived by the D8 algorithm are sparser than those
derived by photogrammetric restitution. Therefore, to obtain accurate
drainage networks, the suitable ow accumulation threshold values
should be determined. Rnyi spectra were calculated for several threshold values (Table 1) and were compared with those for the drainage
networks determined by photogrammetric restitution.
As a prior step to calculate the generalised dimensions or the Rnyi
spectrum, the sandbox method determines the scaling curves for
q [5, 5] for each study basin and DEM resolution (Fig. 4) by considering the abovementioned selected values for R/L in the section method.
This study is limited to this range of q values to avoid instabilities in the
multifractal parameters because higher and lower moment orders may
magnify the inuence of outliers in the measurements as stated by
Zeleke and Si (2005). In addition, some authors, including Lombardo
et al. (2014), have proposed that order moments lower than 3 should
be considered to avoid non-suitable results from the multifractal
analysis. Although the Rnyi spectrum was determined for q [5,
5], their recommendation was followed in this work because only the
fractal dimensions D0, D1 and D2 were used to describe drainage
network morphology. Thus, the scaling curves were linearly t to obtain
the different values of Dq. To obtain the best ts, the linear regressions
were cut between the lower (R/L)lower and upper (R/L)upper limits for
q = 0 that are shown in Table 2. The goodness of the ts (r2) is listed
in the same table. The generalised dimensions were obtained as the
slopes of the linear regressions.
The resemblances between the drainage networks that were
provided by the D8 algorithm and by photogrammetric restitution
were veried by using the Rnyi spectra for each basin and DEM resolution. Table 1 lists the root mean square errors (RMSE) for the considered
ow accumulation thresholds and, in bold, the selected values whose
errors are the lowest. The selected value for each case was lower than
the threshold value for 1% of the maximum ow accumulation, except
for basin 2 for the 10 m DEM resolution for which 1% of the maximum
ow accumulation was the most suitable. The chosen ow accumulation threshold values of basins 1 and 2 are closer to the threshold
value for 1% of the maximum ow accumulation, and represent 0.9%,
1.32% and 0.57% (basin 1) and 0.73%, 0.92% and 0.57% (basin 2) of the
maximum ow accumulation values for the 5, 10 and 25 m DEM resolutions, respectively. These values are derived from Table 1 for each
selected ow accumulation threshold value. However, the same circumstance was not seen for basins 3 and 4, where the selected ow accumulation threshold values are lower than those corresponding to 1% of the
maximum ow accumulation; they were 0.14%, 0.28% and 0.26% (basin
3) and 0.075%, 0.15% and 0.28% (basin 4) of the maximum ow accumulation threshold values for the 5, 10 and 25 m DEM resolutions,

A.B. Ariza-Villaverde et al. / Geomorphology 241 (2015) 243254

Fig. 3. Comparison of the drainage networks for a threshold value of 1% of the maximum ow accumulation against those provided by photogrammetric restitution.

247

1318
327
52
0.1439
0.1551
0.1509
0.1323
0.3028
0.7300
0.1248
0.2736
0.7300
0.1199
0.2382
0.7300
0.0951
0.1693
0.7256
0.0791
0.1662
0.7018
0.0637
0.1600
0.6435
0.0486
0.1501
0.5384
0.0269
0.1225
0.3535
0.0381
0.0252 (0.15%)
0.2473
Root mean square error (RMSE) basin 4
DEM (5 m)
0.0862 0.0701
0.0638
DEM (10 m) 0.0861 0.0381
0.0352
DEM (25 m) 0.0828 0.0796 (0.28%) 0.0889

0.0590
0.0332
0.1087

0.0121 (0.075%)
0.0588
0.2489

1401
352
57
0.1841
0.1542
0.2100
0.1044
0.3753
0.4454
0.0983
0.3596
0.4491
0.0798
0.3217
0.4439
0.0559
0.2728
0.4393
0.0472
0.2461
0.4389
0.0284
0.1851
0.4029
0.0186
0.1348
0.3283
0.0107 (0.14%)
0.0744
0.2545
0.0529
0.0523
0.0933
Root mean square error (RMSE) basin 3
DEM (5 m)
0.0778 0.0591
0.0545
DEM (10 m) 0.0816 0.0795
0.07889
DEM (25 m) 0.0423 0.0182 (0.26%) 0.0599

0.0538
0.0735
0.0844

0.0417
0.0112 (0.28%)
0.2433

887
226
35
0.0388
0.0293
0.0429
0.0414
0.1253
0.8280
0.0387 (0.73%)
0.1185
0.8196
0.0460
0.1059
0.8056
0.0531
0.0990
0.7965
0.0537
0.0955
0.7412
0.0678
0.0583
0.5540
0.0799
0.0324 (0.92%)
0.4474
0.0824
0.0389
0.2523
0.0913
0.0691
0.0682
0.0914
0.0695
0.0365
Root mean square error (RMSE) basin 2
DEM (5 m)
0.1092 0.0926
0.0925
DEM (10 m) 0.0744 0.0715
0.0712
DEM (25 m) 0.0658 0.0410
0.0289 (0.57%)

0.0893
0.0598
0.1088

1085
275
44
0.0659
0.0387
0.0618
0.0277
0.3048
0.6360
0.0175 (0.9 %)
0.2735
0.5428
0.0344
0.2674
0.4995
0.0565
0.2558
0.4936
0.0618
0.1838
0.4853
0.1033
0.1134
0.4813
0.1217
0.0572 (1.32%)
0.4689
0.1704
0.0770
0.4089
0.2961
0.2203
0.1430
0.3700
0.2911
0.0260 (0.57%)
Root mean square error (RMSE) basin 1
DEM (5 m)
0.4087 0.3975
0.3829
DEM (10 m) 0.3387 0.3067
0.3040
DEM (25 m) 0.1981 0.1137
0.0724

0.2401
0.1481
0.2803

50
10

15

20

25

100

200

300

400

500

600

700

800

900

1%

1% of the
maximum ow
accumulation

A.B. Ariza-Villaverde et al. / Geomorphology 241 (2015) 243254


Table 1
Selected ow accumulation threshold values determined from the root mean square error (RMSE). The values in parentheses represent the percentage of the maximum ow accumulation value for each selected threshold value. DEM: digital elevation model.

248

respectively. The cells or pixels of the denser basins 3 and 4 required


lower ow accumulations to be included in the drainage network
(Table 3). The opposite situation was found for the sparse basins 1
and 2.
According to Table 1, RMSE decreased until it reached the lower
value, and then it increased. Therefore, these lower RMSE values determined the appropriate ow accumulation threshold values. RMSE
that corresponded to the selected threshold values also increased as
the DEM resolution decreased for basins 3 and 4; however, it was not
found for basins 1 and 2 (Table 1). This is due to the lower drainage density of basins 1 and 2 (Table 3). Table 1 also shows that the chosen ow
accumulation threshold values signicantly varied for basins 1 and 2, as
the DEM resolution was lower.
Fig. 5 shows the Rnyi spectra obtained for the D8 algorithm by
considering the suitable threshold ow accumulation values and photogrammetric restitution drainage networks. The spectra had decreasing
functions, D0 b D1 b D2 (Table 1), which conrms the multifractal nature
for all of the cases. Regarding the extremely dense drainage network
distribution of basin 4, the corresponding Rnyi spectra for the D8 algorithm and photogrammetric restitution overlap for the 5 m DEM resolution, conrming their morphologic similarity. Nevertheless, for the 10
and 25 m DEM resolutions, the spectra are not overlapped. As a consequence, high DEM resolutions are more suitable for dense networks.
The dense drainage network distribution (basin 3) showed a good overlap of the Rnyi spectra for the 10 m DEM resolution. In the case of the
sparser drainage networks (basins 1 and 2), most parts of the Rnyi
spectra calculated for the 5 and 25 m DEM resolutions overlap, which
shows that the drainage networks have a strong resemblance.
Table 2 lists the multifractal dimensions provided by the Rnyi spectra for the study basins and DEM resolutions. D0 is higher for all of the
drainage networks determined by photogrammetric restitution, with a
higher degree of lling space. These networks have a greater density
of streams than those generated by the D8 algorithm (Fig. 5). It can be
observed that the drainage networks obtained by photogrammetric restitution present higher D1 values in most cases (Table 2) and show more
homogeneous singularity distributions than the drainage networks
from the D8 algorithm. Regarding the correlation dimensions, the photogrammetric restitution drainage networks present higher values of D2
in most cases and show more uniform distributions than those generated by the D8 algorithm.
Fig. 6 shows the drainage networks that were simulated by the D8
algorithm for the suitable threshold ow accumulation values and the
drainage networks determined by photogrammetric restitution. Both
types of drainage networks are quite similar at all DEM resolutions.
However, there is an evident similarity for the low DEM resolutions
and the sparse drainage networks (basins 1 and 2), which is conrmed
by the low RMSE values for both cases (Table 1). Regarding the dense
drainage networks (basins 3 and 4), the resemblance to photogrammetric drainage networks is much closer for the high DEM resolutions with
lower RMSE (Table 1).
4.2. Analysis of the relationship between DEM resolutions and terrain
attributes
Fig. 6 suggests that low DEM resolutions produce stream networks
with channel heads located farthest from the main courses. However,
this effect is reduced by selecting a suitable ow accumulation threshold value with the multifractal analysis. A suitable option is to determine the links between DEM resolution and terrain attributes such as
the ruggedness index Rn, relief ratio Rr and drainage density
Dd (Table 3). Other variables such as climate and lithology control
drainage network morphology (Tucker and Slingerland, 1997; Tucker
and Bras, 2000; Gaudio et al., 2006; Dombrdi et al., 2007). In the four
study basins, however, the inuence of the latter factors is negligible because all of them have a continental Mediterranean climate and metamorphic bedrock (slate).

A.B. Ariza-Villaverde et al. / Geomorphology 241 (2015) 243254

Fig. 4. Scale curves from the Sandbox multifractal analysis.

249

250

A.B. Ariza-Villaverde et al. / Geomorphology 241 (2015) 243254

Table 2
Multifractal dimensions for each study basin and DEM resolution.
Multifractal parameters

Basin 1 DEM 5 m
Photogrammetric restitution
D8 algorithm
Basin 1 DEM 10 m
Photogrammetric restitution
D8 algorithm
Basin 1 DEM 25 m
Photogrammetric restitution
D8 algorithm
Basin 2 DEM 5 m
Photogrammetric restitution
D8 algorithm
Basin 2 DEM 10 m
Photogrammetric restitution
D8 algorithm
Basin 2 DEM 25 m
Photogrammetric restitution
D8 algorithm
Basin 3 DEM 5 m
Photogrammetric restitution
D8 algorithm
Basin 3 DEM 10 m
Photogrammetric restitution
D8 algorithm
Basin 3 DEM 25 m
Photogrammetric restitution
D8 algorithm
Basin 4 DEM 5 m
Photogrammetric restitution
D8 algorithm
Basin 4 DEM 10 m
Photogrammetric Restitution
D8 algorithm
Basin 4 DEM 25 m
Photogrammetric restitution
D8 algorithm

D0

D1

D2

R/Llower

R/Lupper

R2

Net points

Maximum number
of channel order

1.4208
1.4183

1.3438
1.3513

1.2941
1.3117

3.0
2.9

1.39
1.39

0.9992
0.9992

2596
2473

4
4

1.4402
1.3799

1.3944
1.3434

1.3554
1.3128

3.0
2.9

1.39
1.39

0.9988
0.9937

1280
783

4
4

1.4564
1.4894

1.4149
1.4495

1.3803
1.4191

2.8
3.6

1.39
1.39

0.9984
0.9980

509
412

4
4

1.6918
1.6651

1.6826
1.6451

1.6738
1.6303

2.8
2.5

1.39
1.39

0.9992
0.9991

3032
2409

3
3

1.6856
1.6545

1.6795
1.6369

1.6723
1.6270

2.9
2.4

1.39
1.39

0.9994
0.9992

1509
765

3
3

1.6961
1.7046

1.6907
1.6817

1.6858
1.6622

2.8
2.8

1.39
1.39

0.9990
0.9989

597
407

3
4

1.6578
1.6624

1.6174
1.6334

1.5995
1.6140

3.6
3.5

1.39
1.39

0.9987
0.9992

6805
7161

4
5

1.7363
1.7358

1.7126
1.7062

1.6951
1.6850

3.7
3.1

1.39
1.39

0.9987
0.9993

3389
2813

4
4

1.7595
1.7454

1.7375
1.7292

1.7224
1.7204

3.4
3.5

1.39
1.39

0.9984
0.9986

1320
950

4
5

1.6761
1.6857

1.6682
1.6721

1.6599
1.6613

4.3
4.2

1.39
1.39

0.9993
0.9990

7584
8632

4
5

1.6779
1.6699

1.6713
1.6506

1.6646
1.6355

4.4
4.0

1.39
1.39

0.9994
0.9990

3759
2422

4
4

1.6903
1.6146

1.6868
1.5927

1.6835
1.5777

4.0
3.0

1.39
1.39

0.9990
0.9982

1468
732

4
4

Dd varies with DEM resolution (Table 3). In the case of the denser
drainage network distributions (basins 3 and 4), the D8 algorithm provides higher Dd values as the DEM resolution increases. However, this
relationship was not observed for sparser drainage network (basins 1
and 2).

Another relevant attribute is Rn, which combines the relief, H, and


Dd (reference needed). Table 3 shows that Rn diminishes with decreasing DEM resolution for denser drainage networks (basins 3 and 4),
whereas the opposite occurs for basins 1 and 2 with higher Rn values
for the 25 m DEM resolution (Table 3). Rn increases with increasing

Table 3
Geomorphologic indexes of the study basins for 5, 10 and 25 m DEM resolutions. L: total stream length; A: area of the basin; Lb: linear distance between the mouth of the basin and the
point most distant from the mouth; Hmin: minimum height of the basin; Hmax: maximum height of the basin; H: difference in the maximum and minimum height; Dd: drainage density;
Rn: ruggedness index; Rr: relief ratio.
Geomorphologic indexes obtained for drainage networks provided by D8 algorithm
L
(km)

A
(km2)

Lb
(km)

Hmin
(m)

Hmax
(m)

H
(m)

Dd
(km1)

Basin 1
DEM 5 m
DEM 10 m
DEM 25 m

10.30
8.80
11.40

2.71
2.71
2.71

2.50
2.50
2.50

241
242
241

584
583
582

343
341
341

Basin 2
DEM 5 m
DEM 10 m
DEM 25 m

9.90
8.53
11.38

2.24
2.24
2.24

2.20
2.20
2.20

154
155
157

544
545
543

Basin 3
DEM 5 m
DEM 10 m
DEM 25 m

30.90
23.16
22.21

3.50
3.50
3.50

2.60
2.60
2.60

391
390
392

Basin 4
DEM 5 m
DEM 10 m
DEM 25 m

38.00
27.05
20.48

3.30
3.30
3.30

3.60
3.60
3.60

163
163
163

Rn

Rh

3.801
3.247
4.208

1.3036
1.1073
1.4348

0.1372
0.1364
0.1364

390
390
386

4.419
3.808
5.080

1.7209
1.4851
1.9610

0.1773
0.1773
0.1755

556
556
555

165
166
163

8.829
6.617
6.346

1.4567
1.0984
1.0344

0.0628
0.0638
0.0627

274
274
274

111
111
109

11.515
8.197
6.206

1.2801
0.9099
0.6765

0.0308
0.0308
0.0303

A.B. Ariza-Villaverde et al. / Geomorphology 241 (2015) 243254

Fig. 5. Rnyi or generalised dimension spectra for the simulated D8 and photogrammetric restitution drainage networks.

251

252
A.B. Ariza-Villaverde et al. / Geomorphology 241 (2015) 243254

Fig. 6. Drainage network map generated with the D8 algorithm and by photogrammetric restitution for the selected ow accumulation threshold value.

A.B. Ariza-Villaverde et al. / Geomorphology 241 (2015) 243254

Dd for all basins. This nding is supported by the positive relationship


between relief and Dd for landscapes where inltration-excess overland
ow is the primary runoff generation mechanism (Tucker and Bras,
1998), as observed in the basins studied.
Rr measures the overall gradient across a basin based on H and the
basin length Lb. As can be observed in Table 3, those basins with lower
values of Dd (basins 1 and 2) have higher values of Rr. However, the
opposite case occurs for basins 3 and 4 with higher values of Dd. This
nding is supported by the negative relationship between H and Dd
(Table 3). In addition, Rr does not have the same value for the tested
DEM resolutions in all the basins because pixel size affects the H
value to some extent.
5. Conclusions
In this work, a multifractal analysis was applied to assess the effects
of DEM resolution on the ow accumulation threshold value for generating drainage networks and to improve the accuracy of the results of
the D8 algorithm. Based on the results obtained, the use of high DEM
resolutions provides accurate drainage networks and minimises the
error, especially when the drainage network is dense. Nevertheless, in
the case of sparse drainage networks, the inuence of the DEM resolution was less signicant. Moreover, the use of multifractal analysis to determine a suitable threshold ow accumulation value according to the
DEM resolution allows us to obtain more precise drainage networks.
It is important to consider the geomorphologic indexes of drainage
basins for selecting a suitable ow accumulation threshold value. Catchments with high relief ratios present high drainage densities and lower
ow accumulation threshold values and vice versa. In addition, DEM
resolution inuences the suitable ow accumulation threshold value
and therefore network morphology. When a drainage network is
dense, the appropriate ow accumulation threshold value is lower
than 1% of the maximum ow accumulation, and it does not vary greatly
with the DEM resolution. However, in the case of lower drainage density
and higher relief ratio, the ow accumulation threshold value varies signicantly with DEM resolution and is approximately 1% of the maximum ow accumulation, especially when the DEM resolution is high.
References
Ariza-Villaverde, A.B., Jimnez-Hornero, F.J., Gutirrez de Rav, E., 2013. Multifractal analysis applied to the study of the accuracy of DEM-based stream derivation. Geomorphology 197, 8595.
Babbar-Sebens, M., Barr, R.C., Tedesco, L.P., Anderson, M., 2013. Spatial identication and
optimization of upland wetlands in agricultural watersheds. Ecol. Eng. 52, 130142.
Badii, R., Politi, A., 1984a. Hausdorff dimension and uniformity factor of strange attractors.
Phys. Rev. Lett. 52 (19), 16611664.
Badii, R., Politi, A., 1984b. Statistical description of chaotic attractors. J. Stat. Phys. 40,
725750.
Badii, R., Politi, A., 1985. Statistical description of chaotic attractors: the dimension function. J. Stat. Phys. 40, 725750.
Beven, K.J., Kirkby, M.J., 1979. A physically-based variable contributing area model of
basin hydrology. Hydrol. Sci. Bull. 24, 4369.
Cai, t., Li, Q., Yu, M., Lu, G., Cheng, L., Wei, X., 2012. Investigation into the impacts of landuse change on sediment yield characteristics in the upper Huaihe River basin, China.
Phys. Chem. Earth 5354, 19.
Chaubey, I., Cotter, A.S., Costello, T.A., Soerens, T.S., 2005. Effect of DEM data resolution on
SWAT output uncertainty. Hydrol. Process. 19, 621628.
Costa-Cabral, M., Burges, S.J., 1994. Digital elevation model networks (DEMON): a model
of ow over hillslopes for computation of contributing and dispersal areas. Water
Resour. Res. 30 (6), 16811692.
Davis, A., Marshak, A., Wiscombe, W., Cahalan, R., 1994. Multifractal characterization of
non stationarity and intermittency in geophysical elds: observed, retrieved or simulated. J. Geophys. Res. 99, 80558072.
De Bartolo, S., Maiolo, G., Veltri, M., Veltri, P., 1995. Sulla caratterizzazione multifrattale
dlie reti uviali (in Italian). Idrotecnica 6, 329340.
De Bartolo, S.G., Grabiele, S., Gaudio, R., 2000. Multifractal behaviour river networks.
Hydrol. Earth Syst. Sci. 4, 105112.
De Bartolo, S.G., Gaudio, L., Gabriele, S., 2004. Multifractal analysis of river network:
sandbox approach. Water Resour. Res. 40, W02201 (http://dx.doi.org/10.1029/
2003WR002760).
De Bartolo, S.G., Veltri, M., Primavera, L., 2006a. Estimated generalized dimensions of river
networks. J. Hydrol. 322 (14), 181191.

253

De Bartolo, S.G., Primavera, L., Gaudio, R., D'Ippolito, A., Veltri, M., 2006b. Fixed mass
multifractal analysis of river networks and braided channels. Phys. Rev. E 74 (2),
026101.
Dehvari, A., Heck, R.J., 2013. Effect of LIDAR derived DEM resolution on terrain attributes,
stream characterization and watershed delineation. Int. J. Agric. Crop Sci. 6 (13),
949967.
Deng, Y., Wilsong, J.P., Bauer, B.O., 2007. DEM resolution dependencies of terrain attributes across a landscape. Int. J. Geogr. Inf. Sci. 21 (12), 187213.
Dombrdi, E., Timr, G., Bada, G., Cloetingh, S., Horvth, F., 2007. Fractal dimension estimations of drainage network in the CarpathianPannonian system. Glob. Planet.
Chang. 58 (14), 197213.
Falconer, K.J., 1990. Fractal Geometry: Mathematical Foundations and Applications. 2nd
ed. John Wiley, Chichester, UK.
Falorni, G., Teles, V., Vivoni, E.R., Bras, R.L., Amaratunga, K.S., 2005. Analysis and characterization of the vertical accuracy of digital elevation models from the Shuttle Radar Topography Mission. J. Geophys. Res. 110, F02005.
Felicsimo, A.M., 1994. Modelos Digitales de Terreno. Introduccin y aplicaciones en las
ciencias ambientales (Digital Elevation Model: Introduction and Applications to Environmental Sciences). Pentalfa, Oviedo, Spain.
Freeman, T.G., 1991. Calculating catchment area with divergent ow based on a regular
grid. Comput. Geosci. 17 (3), 413422.
Gallant, J.C., Wilson, J.P., 1996. Tapes-g: a grid-based terrain analysis program for the environmental sciences. Comput. Geosci. 22 (7), 713722.
Gaudio, R., De Bartolo, S., Primavera, L., Vertri, M., Gabriele, S., 2004. Procedures in
Multifractal Analysis of River Networks: A State-of-the-Art Review. 15 (2), 228237.
Gaudio, R., De Bartolo, S.G., Primavera, L., Grabiele, S., Veltri, M., 2006. Lithologic control
on the multifractal spectrum of river networks. J. Hydrol. 327 (34), 365375.
Gericke, A., Venohr, M., 2012. Improving the estimation of erosion-related suspended
solid yields in mountainous, non-alpine river catchments. Environ. Model Softw. 37,
3040.
Grassberger, P., 1983. Generalized dimensions of strange attractors. Phys. Rev. Lett. 97 (6),
227230.
Grassberger, P., Procaccia, I., 1983. Characterization of strange attractors. Phys. Rev. Lett.
50, 346349.
Jones, R., 2002. Algorithms for using a DEM for mapping catchment areas of stream sediment samples. Comput. Geosci. 28 (9), 10511060.
Julian, J.P., Elmore, A.J., Guinn, S.M., 2012. Channel head locations in forested watersheds
across the mid-Atlantic United States: a physiographic analysis. Geomorphology
177178, 194203.
Kenward, T., Lettenmaier, D.P., Wood, E.F., Fielding, E., 2000. Effects of digital elevation
model accuracy on hydrologic predictions. Remote Sens. Environ. 74 (3), 432444.
Lea, N.L., 1992. An Aspect Driven Kinematic Routing Algorithm, in Overland Flow: Hydraulics and Erosion Mechanics. Chapman and Hall, New York.
Lombardo, F., Volpi1, E., Koutsoyiannis, D., Papalexiou, S.M., 2014. Just two moments! A
cautionary note against use of high-order moments in multifractal models in hydrology. Hydrol. Earth Syst. Sci. 18, 243255.
Lpez-Vicente, M., Poesen, J., Navas, A., Gaspar, L., 2013. Predicting runoff and sediment
connectivity and soil erosion by water for different land use scenarios in the Spanish
Pre-Pyrenees. Catena 102, 6273.
Mach, J., Mas, F., Sagus, F., 1995. Two representations in multifractal analysis. J. Phys. A
28, 56075622.
Maidment, D.R., 2002. ArcHydro: GIS for Water Resources. ESRI Press, Redlands.
Martz, L.W., Garbrecht, J., 1992. Numerical denition of drainage network and
subcatchment areas from digital elevation models. Comput. Geosci. 18 (6), 747761.
Martz, L.W., Garbrecht, J., 1998. The treatment of at areas and depressions in automated
drainage analysis of raster digital elevation models. Hydrol. Process. 12 (6), 843855.
McMaster, K.J., 2002. Effects of digital elevation model resolution on derived stream
network positions. Water Resour. Res. 38 (4). http://dx.doi.org/10.1029/
2000WR000150.
Montgomery, D., Dietrich, W., 1988. Where do channels begin? Nature 336 (6196), 232234.
Montgomery, D.R., Dietrich, W.E., 1992. Channel initiation and the problem of landscape
scale. Science 255 (5046), 826830.
Moore, I.D., Grayson, R.B., 1991. Terrain-based catchment partitioning and runoff prediction using vector elevation data. Water Resour. Res. 27 (6), 11771191.
O'Callaghan, J., Mark, D.M., 1984. The extraction of drainage networks from digital elevation data. Comput. Vis. Graph. Image Process. 28 (3), 323344.
Orlandini, S., Moretti, G., 2009. Comment on Global search algorithm for nondispersive
ow path extraction by Kyungrock Paik. J. Geophys. Res. 114, F04004.
Orlandini, S., Moretti, G., Franchini, M., 2003. Path-based methods for the determination
of nondispersive drainage directions in grid-based digital elevation models. Water
Resour. Res. 39 (6), 1144. http://dx.doi.org/10.1029/2002WR001639.
Orlandini, S., Tarolli, P., Moretti, G., Dalla Fontana, G., 2011. On the prediction of channel
heads in a complex alpine terrain using gridded elevation data. Water Resour. Res.
47, W02538. http://dx.doi.org/10.1029/2010WR009648.
Orlandini, S., Moretti, G., Gavioli, A., 2014. Analytical basis for determining slope lines in
grid digital elevation models. Water Resour. Res. 50 (1), 526539.
Paik, K., 2008. Global search algorithm for nondispersive ow path extraction. J. Geophys.
Res. 113 (F4), F04001. http://dx.doi.org/10.1029/2007JF000964.
Pawelzik, K., Schuster, H.G., 1987. Generalized dimensions and entropies from a measured times series. Phys. Rev. A 35 (1), 481484.
Quinn, P.F., Beven, K.J., Chevalier, P., Planchon, O., 1991. The prediction of hillslope ow
paths for distributed hydrological modeling using digital elevation models. Hydrol.
Process. 5, 5979.
Rigon, R., Rinaldo, A., Rodriguez-Iturbe, I., Ijjasz-Vasquez, E., Bras, R.L., 1993. Optimal
channel networks: a framework for the study of river basin morphology. Water
Resour. Res. 29 (6), 16351646.

254

A.B. Ariza-Villaverde et al. / Geomorphology 241 (2015) 243254

Rinaldo, A., Rodriguez-Iturbe, I., Rigon, R., Bras, R.L., Ijjasz-Vasquez, E., Marani, A., 1992.
Minimum energy and fractal structures of drainage networks. Water Resour. Res.
28 (9), 21832195.
Rinaldo, A., Rodriguez-Iturbe, I., Rigon, R., Ijjaszvasquez, E., Bras, R.L., 1993. Self-organized
fractal river networks. Phys. Rev. Lett. 70 (6), 822825.
Rivas-Martnez, S., 1987. Map of Vegetation Series of Spain. National Institute for Nature
Conservation, Madrid.
Rodriguez, F., Bocher, E., Chancibault, K., 2013. Terrain representation impact on
periurban catchment morphological properties. J. Hydrol. 485, 5467.
Russel, D., Hanson, J., Ott, E., 1980. Dimension of strange attractors. Phys. Rev. Lett. 45,
11751178.
Saa, A., Gasco, G., Grau, J.B., Anton, J.M., Tarquis, A.M., 2007. Comparison of gliding box and
box counting methods in river network analysis. Nonlinear Process. Geophys. 14,
603613.
Saunders, W., 1999. Preparation of DEMs for Use in Environmental Modelling Analysis,
1999 ESRI User Conference. ESRI Online, San Diego, CA.
Srensen, R., Seibert, J., 2007. Effects of DEM resolution on the calculation of topographical
indices: TWI and its components. J. Hydrol. 347 (12), 7989.
Tarboton, D.G., 1997. A new method for the determination of ow directions and upslope
areas in grid digital elevation models. Water Resour. Res. 33 (2), 309319.
Tarboton, D.G., Bras, R.L., Rodriguez-Iturbe, I., 1988. The fractal nature of river networks.
Water Resour. Res. 24 (8), 13171322.
Tl, T., Flp, ., Vicsek, T., 1989. Determination of fractal dimensions for geometrical
multifractals. Physica A 159 (2), 155166.
Thieken, A.H., Lcke, A., Diekkrger, B., Richter, O., 1999. Scaling input data by GIS for
hydrological modelling. Hydrol. Process. 13, 611630.
Tribe, A., 1992. Automated recognition of valley lines and drainage networks from grid
digital elevation models: a review and a new method. J. Hydrol. 139 (14), 263293.
Tucker, G.E., Bras, R.L., 1998. Hillslope processes, drainage density, and landscape morphology. Water Resour. Res. 34 (10), 27512764.

Tucker, G.E., Bras, R.L., 2000. A stochastic approach to modelling the role of rainfall variability in drainage basin evolution. Water Resour. Res. 36 (7), 19531964.
Tucker, G.E., Slingerland, R., 1997. Drainage basin responses to climate change. Water
Resour. Res. 33 (8), 20312047.
Turcotte, R., Fortin, J.-P., Rousseau, A.N., Massicotte, S., Villeneuve, J.-P., 2001. Determination of the drainage structure of a watershed using a digital elevation model and a
digital river and lake network. J. Hydrol. 240 (34), 225242.
Vaze, J., Teng, J., Spencer, G., 2010. Impact of DEM accuracy and resolution on topographic
indices. Environ. Model Softw. 25 (10), 10861098.
Vicsek, T., 1990. Mass multifractal. Physica A 168, 490497.
Vicsek, T., Family, F., Meakin, P., 1990. Multifractal geometry of diffusion-limited aggregates. Europhys. Lett. 12 (3), 217222.
Walker, J.P., Willgoose, G.R., 1999. On the effect of digital elevation model accuracy on hydrology and geomorphology. Water Resour. Res. 35 (7), 22592268.
Wolock, D.M., Price, C.V., 1994. Effect of digital elevation model and map scale and data
resolution on a topography-based watershed model. Water Resour. Res. 30 (11),
30413052.
Wu, S., Li, J., Huang, G.H., 2008a. A study on DEM-derived primary topographic attributes
for hydrologic applications: sensitivity to elevation data resolution. Appl. Geogr. 28,
210223.
Wu, W., Fan, Y., Wang, Z., Liu, H., 2008b. Assessing effects of digital elevation model
resolutions on soil-landscape correlations in a hilly area. Agric. Ecosyst. Environ.
126 (34), 209216.
Zeleke, T.B., Si, B.C., 2005. Scaling relationships between saturated hydraulic conductivity
and soil physical properties. Soil Sci. Soc. Am. J. 69 (6), 16911702.
Zhang, W., Montgomery, D.R., 1994. Digital elevation model grid size, landscape representation, and hydrologic simulations. Water Resour. Res. 30 (4), 10191028.

You might also like