10th International Congress on Advances in Civil Engineering, 17-19 October 2012

Middle East Technical University, Ankara, Turkey

Analysis of heavy rainfall events using hydrometeorological

system and remote sensing
A. Onen1, I. Yucel2
1

Department of Civil Engineering, METU, Ankara, Turkey, e143988@metu.edu.tr

Department of Civil Engineering, METU, Ankara, Turkey, iyucel@metu.edu.tr

Abstract
Extreme weather events such as heavy rainfall and flooding are projected to become much more frequent as
climate warms. Hydrologic predictions are critical for decision-making related to flood mitigation and water
resources. Hydrometeorological model forecasts and satellite data can be used for flood forecasting and warning
issues as they provide continuous monitoring of precipitation data both spatially and temporally at high
resolution. Hydrologic model simulations depend heavily on the availability of reliable precipitation estimates.
Difficulties in estimating precipitation impose an important limitation on the possibility and reliability of
hydrologic forecasting and early warning systems. This study investigates the performance of the rainfall
products obtained from the Multi Precipitation Estimates (MPE) and Weather Research and Forecasting (WRF)
model and provide a statistical analysis among them by comparing with gauge rainfall measurements.
Precipitation estimates obtained by using the three dimensional variational (3D-VAR) data assimilation scheme
within WRF system are also used in statistical analysis. Statistical analysis is performed for the selected number
of heavy rainfall events and associated flooding conditions over the mountainous West Black Sea Basin in
Turkey. The MPE and WRF-estimated rainfall showed capabilities in capturing the timing of the flood events
and in some extent spatial distribution and magnitude of the heavy rainfall events. WRF with assimilation
improved area-averaged precipitation forecasts by 12 percent and at some points there exists quantitative match
for heavy rainfall events, which are critical for hydrological forecast.
Keywords: rainfall, hydrology, assimilation, model

1 Introduction
Each year, northwestern region of Turkey faces multiple flood threats especially during spring and summer
seasons. Extensive precipitation during spring and summer seasons causes lethal and economical consequences
and due to lack of reliable prediction systems, the inevitable results occur multiple times every year. In this
study, rainfall estimates derived from Weather Research and Forecasting (WRF) model and Multi Precipitation
Estimates (MPE) algorithm for a data sparse region such as Western Black Sea Region have been evaluated by
comparing with rain gauge observations obtained from Automated Weather Observation System (AWOS).
Extreme rainfall events caused flooding in the region are selected from the severe hydrometeorological events
chart available for each year in Meteorological General Director data base. This study focuses on documenting
performance of WRF model with use of 3D-VAR scheme on heavy rainfall events occurred in West Black Sea
region of Turkey. Similar studies that involve using WRF modeling system with 3D-VAR have been applied in
different parts of the world (Choi et al. (2011), Xiao et al. (2005)).
In this study, 34 rain gauge stations scattered across West Black Sea Region (Figure 1) have been selected to use
for statistical analysis. Statistical analysis is performed by comparing rain gauges with WRF and MPE rainfall
estimates for number of extreme rainfall events selected in this study. Comparisons are made at closest pixels on
both 4-km WRF and MPE grids, which correspond to each rain gauge stations. Time series of single stationpixels have been sorted out from satellite and model domains.

ACE2012

Figure 1. Locations of the study domain (Left) and detailed view (Right) with Rain Gauge Stations (shown with
nodes)
In statistical analysis, time series of observations, WRF and MPE outputs have been re-arranged into 1, 3, 6 and
24 hourly periods for the selected 20 events time spans (Table 1) and inspected with respect to multiple
statistical variables. For each event, corresponding rainfall data from 34 stations, including observation, WRF
with and without assimilation model output and MPE output, have been listed versus event period and
statistically analyzed with respect to 4 different datasets simultaneously. In addition to event based analyses, 2
more approaches have also been made: In second approach, stations have been studied individually to inspect the
reliability of the station readings by associating together same stations datasets from each event selected.
Thirdly, regardless of time and stations, all data from events selected have combined together to generate a
general overview of statistical variance between observation, WRF and MPE outputs. In this paper, as the all
statistical results are inspected, 3 and 24 hourly analyses have been shown to demonstrate the aim of the paper.
Table 1. Selected Events List
Event
No.
1
2
3
4
5
6
7
8
9
10

Start Date

End Date

02.06.00
04.08.02
19.08.02
11.08.04
14.08.04
23.08.04
30.04.05
14.07.05
04.06.07
27.09.08

06.06.00
07.08.02
23.08.02
16.08.04
17.08.04
28.08.04
02.05.05
17.07.05
15.06.07
01.10.08

Event
No.
11
12
13
14
15
16
17
18
19
20

Start Date

End Date

12.07.09
27.07.09
06.09.09
19.09.09
25.06.10
06.07.10
31.08.10
07.10.10
28.05.11
09.06.11

16.07.09
28.07.09
10.09.09
25.09.09
01.07.10
10.07.10
04.09.10
10.10.10
30.05.11
13.06.11

2 Data
In this study, as mentioned, 4 different datasets have been used for statistical analysis. The first dataset,
observations, has been collected from selected observation stations for the selected time spans. The second
dataset, WRF Model is a mesoscale numerical weather prediction system designed to serve both operational
forecasting and atmospheric research needs (Grell et al. (1995)). It features multiple dynamical cores, a 3dimensional variational (3DVAR) data assimilation system, and a software architecture allowing for
computational parallelism and system extensibility. WRF is suitable for a broad spectrum of applications across
scales ranging from meters to thousands of kilometers. As 4-km grid scale has been used for this study, WRF
model outputs are both simulated with and without assimilation scheme and necessary pixels have been picked
out from local domain outputs from pixels containing selected station locations.
The Multi-sensor Precipitation Estimate (MPE) is an instantaneous rain rate product which is derived from the
IR-data of the geo-stationary EUMETSAT satellites by continuous re-calibration of the algorithm with rain-rate
data from polar orbiting microwave sensors (Heinemann et al. (2002)). The algorithm is only suitable in
convective weather situations. Frontal precipitation, especially at warm fronts is very often wrongly located and

A. nen, . Ycel
overestimated. Two quality indicators distributed together with the GRIB-2 MPE product give indications where
the product should be used and where it may be problematic.

3 Statistical Analyses
3.1 Event and Station Based Bias, Error Calculations
For the selected 20 events, areal rainfall distributions for 34 stations and 4 km grid points have been inspected.
The effect of local influences on the performance of WRF model and Satellite-generated precipitation is
determined by investigating the local areal changes in rainfall for each event. As a preliminary statistical
analysis, between each datasets, bias and root mean square error (RMSE) have been calculated on Figures 2 and
3 respectively for 3 and 24 hourly event inspections as assimilated model output (WRF Model AS), nonassimilated model output (WRF Model NOAS) and MPE output (Satellite). Considering all events and as a
general statistical overview, decrease in Bias and RMSE have been observed on WRF AS with respect to WRF
NOAS. Thus, while WRF AS Model has shown a minor effect on amount of precipitation, it leads to desired
model patterns on general precipitation trends. As satellite outputs are inspected, MPE readings, which are
available for events starting from year 2007 (Event 9 20), indicates overall underestimation of precipitation
with respect to observation, thus it results in negative bias and falsely lower RMSE values with respect to WRF.
In most cases, WRF AS Model generates significantly better statistical results than MPE.

Figure 2. Event Based 3-Hourly Bias, RMSE Charts

ACE2012

Figure 3. Event Based 24-Hourly Bias, RMSE Charts

In station based analysis, each 34 stations have been evaluated individually for 20 different event periods to
generate a temporal overview. On Figures 4 and 5, 3 and 24-hourly Bias and RMSE charts are shown for
stations. Like in event based analyses, WRF AS Model shows more reliable improvement in station based
analyses. Especially in 3-hourly analyses, correlation coefficients have shown significantly better results with
respect to observations. As station based and event based analyses would have been commented together,
assimilation in WRF has interpreted the temporal effects better than areal effects. In this research, as summertime precipitation events have been selected for analyses, the corresponding events were convective system
generated and in such cases, fuzziness of areal effects results in lower statistical reliability than temporal effects.

A. nen, . Ycel

Figure 4. Station Based 3-Hourly Bias, RMSE Charts

Figure 5. Station Based 24-Hourly Bias, RMSE Charts

ACE2012
The station based analyses, which consider temporal effects along with the events, show lower error rates than
event based analyses which deal with spatial effect within the domain for WRF outputs (Table 2). Uncertainty in
spatial variability over the domain increases significantly higher than that of temporal effect because of the
characteristics of the rain which are in convective origin. Numerical atmospheric models poorly parameterize
convective activities. Since satellite provides better spatial characteristics of rain in more detail, their mean error
estimates are higher than their temporal effect.
Table 2. Mean RMSE scores (mm) for WRF AS, WRF NOAS and MPE
Event Based

Station Based

WRF AS

WRF NOAS

MPE

WRF AS

WRF NOAS

MPE

1 Hourly

1.696

1.775

1.274

1.495

1.550

1.347

3 Hourly

3.657

3.867

2.921

3.346

3.516

3.011

6 Hourly

5.933

6.176

4.651

5.395

5.656

4.782

24 Hourly

12.131

12.729

10.563

12.288

12.898

10.759

To create a more detailed comparison of performance between WRF AS and WRF NAS, an additional RMSE
based calculation has been performed. For both event based and station based analyses, the improvement
percentage in RMSE per event and per station has been calculated. In event based improvement, the averages of
20 events for different time spans have calculated as; 1 hourly 4.31%, 3 hourly - 5.13%, 6 hourly - 3.72% and
24 hourly 4.21%. Logically, the percentage of improvement should increase with respect to longer time
periods, but the presence of WRF NOAS better events creates a chaotic structure within desired pattern. To
observe the effect of assimilation in more desired detail, an additional calculation has been performed. 9 events
(2, 3, 6, 8, 10, 12, 13, 15 and 17) which shows better results for no-assimilation have been discarded temporarily
to obtain only WRF AS focused improvement in error and updated results are turned out to be; for 1 hourly
7.80%, 3 hourly - 9.19%, 6 hourly - 9.29% and 24 hourly 10.12%.
Same procedure has been followed for 34 stations. On general view, the average error improvement for station
based analysis are computed as; 1 hourly 6.39%, 3 hourly - 4.29%, 6 hourly - 3.81% and 24 hourly 4.08%.
Again to discard the chaotic nature of the pattern explained above and to focus of WRF AS favour on
improvement, 15 stations with better NOAS results (Akcakoca, Sinop, Kocaeli, Sakarya, Duzce, Kastamonu,
Geyve, Acisu-Radar, Boyabat, Caycuma, Arac, Gerede, Kibriscik, Eskipazar, Boludagi and Goynuk) have been
discarded temporarily for the calculations, and the updated improvement averages as follows; 1 hourly 8.99%,
3 hourly 11.39%, 6 hourly - 11.46% and 24 hourly 11.20%. On table 3, average RMSE improvements for
analyses have also been given.
Table 3. Event and Station based time dependent RMSE improvement averages

Analysis Type

Event Based Analysis

Station Based
Analysis

Hourly Time Period

Data Type
1 Hourly

3 Hourly

All

4.31%

5.13%

6 Hourly 24 Hourly
3.72%

4.21%

Excl. NOAS better values

7.80%

9.19%

9.29%

10.12%

All

2.79%

4.29%

3.81%

4.08%

Excl. NOAS better values

8.99%

11.39%

11.46%

11.20%

3.2 Event and Station Based Statistical Analyses

In addition to event and station based Bias, RMSE and R calculations, additional precipitation parameters; POD
(Probability of Detection), FAR (False Alarm Rate) and CSI (Critical Success Index) (Chokngamwong, R. ve L.
Chiu, (2007)) have been calculated for further statistical inspections. These parameters are explained as follows:
POD = A/(A+B),

FAR = C/(A+C),

CSI = A/(A+B+C),

A. nen, . Ycel
Where A is the number of matching precipitation data while both observation and model (WRF and MPE) shows
positive (non-zero) precipitation, B is the number of occurrence where observation shows positive and model
shows zero precipitation and lastly C is the number of occurrence where model shows positive and observation
shows zero precipitation.
While improvements provided by assimilation were given per event and per station basis in previous analyses,
POD, FAR and CSI values (Kidd et al. (2011)) have been evaluated together to trace the change in precipitation
forecast performance for both in WRF Model and MPE Model respectively. As event based 3 and 24-hourly
charts have been inspected on Figure 6, it has been observed that WRF model shows higher POD and lower FAR
values than Satellite model in a more scattered pattern, and as the time span increases from hourly to 24 hourly
periods, desired pattern of significant increase in POD and decrease in FAR have been witnessed. Thus, CSI
values, which is a function of both POD and FAR, converges towards 1, shown within contours. Scattering
among the events occurred because of the spatial impacts, vary from event to event. In addition, WRF calculates
the precipitation based on physical parameterization while satellite algorithm determines the precipitation based
on empirical equation. This causes WRF model to predict rainfall in a more scattered way than the MPE model.
On Figure 7, station based POD, FAR, CSI charts are shown for the same time periods. In these charts, WRF and
Satellite models shows closer pattern and as seen in event based charts, the optimization of results increases with
respect to increase in analysis period. The accumulation of scattered points also shows similar patterns within
WRF and Satellite models respectively. For both event and station based analyses, Satellite model shows higher
FAR and lower CSI values, while POD values give close results when compared to WRF model outputs.

Figure 6. Event Based 3 hourly (Left), 24 hourly (Right) POD, FAR, CSI Charts (Kidd et al. (2011))

Figure 7. Station Based 3 hourly (Left), 24 hourly (Right) POD, FAR, CSI Charts (Kidd et al. (2011))

3.3 Whole Dataset Rainfall Characteristics

Following the event and station based analyses, a combined statistical approach of whole datasets of WRF and
Satellite series has been performed to inspect their relationship on a more generalized perspective. Firstly the
correlation of models with respect to observation has been determined on again 3 and 24-hourly time periods. On

ACE2012
Figures 8 and 9, the scatter charts of Observation versus WRF AS, WRF NOAS and Satellite for 3, 24 hourly
periods are shown with the correlation coefficients and equations.
As the charts have been inspected, while there is no correlation can be seen on 1-hourly distribution, on 3 and 24
hourly distributions, trendlines shows a closer pattern to 1:1 axes, resulting in higher correlation values. Also
WRF AS models higher R output on 3 and 24 hourly analyses validates the previous statistical calculations.
Despite the higher R value of WRF outputs, Satellite outputs show visually better scatter distributions vs.
observation and thus charts generally imply WRF model simulating over-estimation while satellite simulating
under-estimation of observations. Also as predicted, WRF AS generates better correlation result than WRF
NOAS.

Figure 8. Scatter Diagrams for 3-Hourly Periods

Figure 9. Scatter Diagrams for 24-Hourly Periods

After general overview of correlation has been generated, for the combined dataset, an extensive statistical
precipitation analysis has been performed. In addition to previously mentioned POD, FAR, CSI calculations,
Equitable Threat Score (ETS) and its bias (ETS Bias) properties (Lee et al., (2004)) have been determined for
several threshold limits. ETS and ETS Bias are defined as below:
ETS = (A-H)/(A+B+C-H),

where H = (A+B)(A+C)/(A+B+C+D),

ETS Bias =(A+B)/(A+C),

Where parameters A, B and C are same as defined in previous section and D is the number of occurrence where
both observation and model shows zero precipitation. In these score calculations, the performances of WRF and
MPE simulations of matching observed rainfall for different amounts of precipitation have been observed. 3Hourly ETS, ETS Bias threshold charts are shown in Figure 10; 24-Hourly ETS, ETS Bias threshold charts are
shown in Figure 11. On a general overview, with the increase of threshold limits, decrease in ETS acts as the
logical behaviour for the corresponding analyses. For 3 and 24-hourly analyses, WRF AS output shows higher
statistical success than WRF NOAS. As a validation of previous statements, Satellite model results in less than 1
ETS Bias scores also underlines the general under-estimation behaviour of MPE.

A. nen, . Ycel

Figure 10. 3-Hourly Whole Data Set Analyzed ETS, ETS Bias Charts

Figure 11. 24-Hourly Whole Data Set Analyzed ETS, ETS Bias Charts

ACE2012

4 Summary and Conclusion

In this study, several statistical analyses are performed between rainfall estimates obtained from WRF and MPE
and rain gauge observations for number of events and stations across West Black Sea Region in Turkey. The
performance of WRF model via 3DVAR data assimilation scheme is tested against the observation. In addition
model results are comparatively evaluated with satellite rainfall estimates. The main results are summarized as
follows:
1) Overally, WRF model with and without assimilation generates an overestimation trend against
observations, while MPE (Satellite) model significantly underestimates the precipitation.
2) In WRF model, the impact of assimilation generally shows improvement for the events and stations.
3) Provided improvements on statistical analyses such as bias, rmse, r, POD, FAR, CSI and ETS show that
the skill of WRF model with assimilation is generally better than WRF NOAS and MPE. Errors with
data assimilation are reduced up to 10% for event based, and up to 12% for station based analyses.
4) WRF Model provides better error estimates temporally while MPE error estimate performance acts
better spatially, however WRF model outputs show generally more skill in capturing temporal and
spatial rain characteristics than MPE.

References
Choi H., Ha J., Lee D., Kuo Y.(2011), Analysis and Simulation of Mesoscale Convective Systems
Accompanying Heavy Rainfall: The Goyang Case, Asia-Pacific J. Atmos. Sci..
Chokngamwong, R. , Chiu, L. (2007), Thailand Daily Rainfall and Comparison with TRMM Products, Journal
of Hydrometeorology vol. 9, Pages 256-266.
Grell, G. A., J. Dudhia, and D. R. Stauffer, 1995: A description of the fifth generation Penn State/NCAR
mesoscale model (MM5). NCAR Tech. Note NCAR/TN-398+STR, 138 pp.
Heinemann T., Lattenzio A., Roveda F. (2002), The Eumetsat Multi Sensor Precipitation Estimate (MPE),
Eumetsat
Kidd, C., Bauer, P., Turk, J., Huffman, G.J., Joyce, R., Hsu, K-L. and Braithwaite, D. (2011), Inter-comparison
of high-resolution precipitation products over northwest Europe, Journal of Hydrometeorology Early Online
Release.
Lee, S. , Lee, D., Chang, D. (2004), Impact of Horizontal Resolution and Cumulus Parameterization Scheme on
the Simulation of Heavy Rainfall Events over the Korean Peninsula, Advances in Atmospheric Sciences, Vol.
28, No. 1, Pages 1-15
Xiao, Q., Kuo, Y.-H., Sun, J., Lee, W.-C., Lim, E., Guo, Y.-R., Barker, D.M. (2005), Assimilation of Doppler
radar observations with a regional 3D-VAR system: Impact of Doppler velocities on forecast of heavy
rainfall case, J. Appl. Meteor., 44, pp. 768-788.

10