Food Control 80 (2017) 151e161

Contents lists available at ScienceDirect

Food Control
journal homepage: www.elsevier.com/locate/foodcont

The performance of food safety management systems in the

raspberries chain
Andreja Rajkovic a, b, *, Nada Smigic a, Ilija Djekic a, Dragana Popovic a, Nikola Tomic a,
Nada Krupezevic a, Mieke Uyttendaele b, Liesbeth Jacxsens b
a
Department of Food Safety and Quality Management, Faculty of Agriculture e University of Belgrade, Nemanjina 6, 11080 Zemun-Belgrade, Serbia
b
Laboratory of Food Microbiology and Food Preservation, Department of Food Safety and Quality, Faculty of Bioscience Engineering e Ghent University,
Coupure Links, 9000 Ghent, Belgium

a r t i c l e i n f o a b s t r a c t

Article history: The Food Safety Management System e Diagnostic Instrument (FSMS-DI) was used to determine context
Received 18 February 2017 factors, core control and assurance activities, as well as food safety outputs for three Global G.A.P.
Received in revised form certified and six Non Global G.A.P. certified orchard raspberries farms, and eight cold stores (all of them
26 April 2017
having certified FSMS systems, often in combination with BRC, IFS and/or FSSC 22000 standards).
Accepted 28 April 2017
Available online 4 May 2017
Examined orchard farms operate at moderate to high-risk context. High risk is mainly related to the
microbiological and pesticides contamination of raspberries and open cultivation system, which can
provoke additional contaminations (e.g. bird droppings). However they differed in chain and organiza-
Keywords:
Food safety management system
tional characteristics. Non Global G.A.P. certified orchard farms were mainly characterised by low to basic
Raspberries performance of the FSMS combined with low food safety outputs, while in Global G.A.P. orchard farms
Cold stores moderate performance of FSMS resulted in moderate to advanced food safety outputs. Cold store
Orchard farms companies represent the subsequent link in the raspberries chain, with the raw material food safety risks
directly connected with the orchard farms final product. This is related to the production process of
frozen raspberries without any physical or chemical intervention step which might reduce the level of
potentially present microorganisms or chemicals in raspberries. The core control and assurance activities
in the FSMS present in the cold stores are mainly at medium to high level, resulting in medium to
advance food safety outputs.
© 2017 Elsevier Ltd. All rights reserved.

1. Introduction
The Serbian cultivated berry sector, especially raspberries and
blackberries, has been a driving force in the national agricultural
economic growth for the last three decades. More than 80,000
farms, 250 cold stores, and 100 processing factories are involved in
this sector (USAID, 2008). Republic of Serbia is one of the biggest
producers and exporters of raspberries in the world with the
annual production of over 60,000 tons (70310 tons in 2012, 68458
tons in 2013 and 61,715 tons in 2014). This fruit was grown on
11,996 ha, 12,024 ha and 11,040 ha, respectively (Serbia, 2016). The
total production value in 2014 is estimated at 47 million EUR. Only
* Corresponding author. Department of Food Safety and Quality Management,
Faculty of Agriculture e University of Belgrade, Nemanjina 6, 11080 Zemun-
Belgrade, Serbia.
E-mail address: Andreja.Rajkovic@UGent.be (A. Rajkovic).
about 1 million EUR came from the trade of raspberries on the
domestic market. Raspberries are grown on 70,000 farms, located
mainly in West Serbia, which accounts for nearly 55% of the total
raspberry production; Central Serbia, where 35% of the total rasp-
berry output is produced; and South-West Serbia that produces the
remaining 10% (Leposavic et al., 2013; Nikolic, Ivanovic, Milenkovic,
Milivojevic, & Milutinovic, 2008). Over 90% of Serbian raspberries
are the “Willamette” variety, followed by varieties “Meeker”,
“Tulameen” and “Glen Ample” (Nikolic & Milivojevic, 2015). The
family owned farms’ average size is 1ha (from 20 acres to 5 ha)
(Buric, 2003; Nikolic & Milivojevic, 2015), which is easy to manage,
since raspberry farming requires much manual labour (raspberries
are almost exclusively manually picked). The yield varies from 5 to
20 t/ha depending on the agricultural practice in place, local cli-
matic and weather conditions (Nikolic & Milivojevic, 2015). Sta-
tistical average for the period 2012e2014 was about 5.73 t/ha
(Serbia, 2015).

http://dx.doi.org/10.1016/j.foodcont.2017.04.048
0956-7135/© 2017 Elsevier Ltd. All rights reserved.
152 A. Rajkovic et al. / Food Control 80 (2017) 151e161
Rapid Alert System for Food and Feed (RASFF) of the European
Commission shows currently 70 notifications (mainly alerts)
involving raspberries of different origin, 17 of these indirectly or
directly involving Serbia (RASFF, 2016). Out of 70 notifications, 39
were classified as pathogenic microbial contamination, mainly
norovirus, but also caliciviruses, Escherichia coli and Shigella spp.
Reported outbreaks with raspberries involve also Cyclospora caye-
tanensis (Caceres et al., 1998; Gibbs et al., 2013; Herwaldt & Ackers,
1997). However the viral contamination remains the most promi-
nent, causing the largest number of reported outbreaks
(Bouwknegt et al., 2015; Sarvikivi et al., 2012; Verhaelen et al.,
2013). This in general corresponds to the opinion of 54 experts
who considered bacterial as the most important food safety issue
for fresh produce in general, followed by foodborne viruses,
pesticide residues and mycotoxins (Van Boxstael et al., 2013).
Fresh produce may be contaminated pre-harvest due to irriga-
tion with reclaimed wastewater, crop fertilization with sewage
sludge or faecal pollution of the production areas (Chigor et al.,
2012; Forslund et al., 2010) and pesticides spraying (Stine, Song,
Choi, & Gerba, 2011; Verhaelen et al., 2013). During harvest, food
may be contaminated by workers at the field, as was suspected in
several outbreaks of Norwalk-like virus, and other enteric viruses
from raspberries, leafy vegetables, etc. (Berger et al., 2010;
Cotterelle et al., 2005). Fresh produce may be also contaminated
during postharvest phases, including product handing, preparation
and packaging (Lynch, Tauxe, & Hedberg, 2009; Richards, 2001).
Due to the inherent difficulties in detection of viruses in rasp-
berries, one cannot rely on sampling and end-product analysis for
any effective control measures, therefore the key efforts must be
placed on preventive pre-harvest and post-harvest measures.
An effective Food Safety Management System (FSMS) has to be
in place in order to control different hazards in the fruit/raspberry
chain. It is the obligation for all food producers to have in place an
effective system based on various good practices depending on
their role in the food chain. Good agricultural practice is expected in
primary production (orchards). Fruit processing (cold storage
plants) has to comply with Good Manufacturing Practices (GMP)
and the Hazard Analysis and Critical Control Points (HACCP) prin-
ciples, as outlined in Regulation (EC) 852/2004 and in Serbian
Ordinance on hygiene for producers (Serbia, 2010c). It is important
to note that these requirements are generic regardless of the type
and size of company and therefore should be tailored for the spe-
cific company and its business environment. Management system
effectiveness assessment (including control and assurance activ-
ities) enables discovering of problems which are related to imple-
mentation, as well as tracking down the bottlenecks and possible
ways for the improvements to the system at the specific company. A
tool for the diagnosis of the effectiveness of food safety system was
previously developed, in order to determine control activities and
core assurance activities in a company’s FSMS (Jacxsens et al., 2010;
Luning, Bango, Kussaga, Rovira, & Marcelis, 2008; Luning et al.,
2011; Luning et al., 2009). It is of note that the system output, be-
ing the safety and hygiene of processed fruits or food products, is
not only dependent on the system design and operation, but also on
the context wherein it operates, as this influences the overall risk
(Kirezieva, Nanyunja, et al., 2013). This tool enables one to identify
the riskiness of the company’s contextual situation in relation to its
food safety performance. The diagnostic instrument has been
applied in animal food processing companies, within dairy
(Sampers, Toyofuku, Luning, Uyttendaele, & Jacxsens, 2012) and
meat (Ose
s et al., 2012; Sampers et al., 2010) industry. Within the
European FP7 project Veg-i-Trade, FSMS-DI has been tailored for
the fresh produce sector, including primary production, processing
and trade (Kirezieva, Jacxsens, Uyttendaele, Van Boekel, & Luning,
2013). It has been applied in the mushroom production in South
Africa (Dzingirayi & Korsten, 2016), the primary production of
organic lettuce in Brazil (de Quadros Rodrigues et al., 2014), green
bean farms in Kenya, hot pepper farms in Uganda (Nanyunja et al.,
2015a) and fresh produce export processing companies in Kenya
(Sawe, Onyango, & Njage, 2014).
Based on the hypothesis that Global G.A.P. certification ensures a
better FSMS performance in orchard farms producing raspberries,
and due to the restricted set of requirements, the risk-based
approach and annual third party audits, the first objective of this
study was to get an insight and to compare the status of FSMS in
three Global G.A.P. certified and six Non Global G.A.P. certified
farms in Serbia using FSMS-DI. In addition, a better FSMS perfor-
mance is expected in downstream actor in the raspberries chain, i.e.
the cold stores. Therefore, the aim was to determine FSMS perfor-
mance in eight cold stores (all of them having certified HACCP
systems, often in combination with BRC, IFS and/or FSSC 22000
standards), in view of their contextual situation and food safety
output.
2. Material and methods
2.1. Profile of participants
A total number of 17 food business operators were included in
the study, consisting of nine orchard farms growing raspberries
(coded F1 to F9) and eight cold stores (coded CS1 to CS8) doing
purchasing, freezing, packing and distributing berries. All orchard
farms and cold stores were located in Central and West Serbia,
corresponding to the area with about 90% of raspberry plots in
Serbia. The data related to the practices regarding food safety
management systems were collected in orchard farms and cold
stores with the permission of the business operators.
The breakdown of food businesses that participated in this
survey is presented in Table 1. All orchard farms (F1 to F9) involved
in this study were the suppliers of three cold stores (CS1 to CS3).
The other five cold stores (CS4 to CS8) have purchased raspberries
from other farms.
2.2. Food safety system diagnosis
The Food Safety Management System e Diagnostic Instrument
(FSMS-DI) was developed for each actor in the fresh produce chain
within the European project Veg-i-Trade. FSMS-DI used for orchard
farms consisted of 66 questions (Kirezieva, Jacxsens, et al., 2013;
Kirezieva, Nanyunja, et al., 2013), while FSMS-DI used in cold
stores consisted of 71 questions.
The data were obtained during face-to-face interviews with
owners of the orchard farms or responsible QA (quality assurance)
staff of the cold stores using the structured questionnaire. For each
question/indicator, the interviewees had a possibility to grade a
level that is the most representative for their business. Additional
statements and explanations were provided for each question in
the questionnaire to allow an easier selection of the most repre-
sentative situation. Each interview lasted about 2e3 h and was
followed by an on-site visit to confirm the initial assessment.
2.2.1. Diagnosis of riskiness in context
First part of FSMS-DI represents the context diagnosis with a set
of indicators to assess the risk level of the context factors (being
product, process, organizational and chain environment charac-
teristics). For each indicator, three situational descriptors are given
representing low (situation 1), moderate (situation 2) and high
(situation 3) risk. Regarding product and process characteristics,
low, moderate, and high risk situations correspond to low, mod-
erate and high probability of microbial or chemical contamination,
 A. Rajkovic et al. / Food Control 80 (2017) 151e161 153

Table 1
The characteristics of orchard farms and cold stores involved in the study.

Orchard Implemented and Number of Size of farm Ownership Supplier Cold Distance between farm Implemented and Number of
farms certified standard employees store and cold store certified standard employees

F1 Global G.A.P. 1e49 35ars (one parcel) Independent producer / CS1 <0.5 km HACCP 50e249
F2 Global G.A.P. 1e49 15ars (one parcel) Independent producer / <0.5 km IFS
F3 None 1e49 40ars (fragmented) Independent producer / <0.5 km BRC

F4 Global G.A.P. 50e249 3ha (one parcel) In cooperative with CS2 / CS2 3 km HACCP 50e249
ISO 9001
BRC

F5 None 1e49 1ha (fragmented) Independent producer / CS3 4 km HACCP 50e249

F6 None 1e49 1,5ha (one parcel) Independent producer / 3 km ISO 9001
F7 None 1e49 1ha (fragmented) Independent producer / >10 km BRC
F8 None 1e49 2ha (fragmented) Independent producer / >10 km
F9 None 1e49 45ars (one parcel) Independent producer / >10 km

e e e e e e CS4 e HACCP 50e249

ISO 9001
BRC
IFS

e e e e e e CS5 e HACCP 50e249

e e e e e e CS6 e HACCP 50e249

BRC

e e e e e e CS7 e HACCP 50e249

ISO 9001
BRC
FSSC 22000

e e e e e e CS8 e HACCP 10e49

*
HACCP e Hazard Analysis and Critical Control Points, BRC e British Retail Consortium, IFS e International Food Standard.

growth or survival of pathogens and undesired microorganisms
respectively. For organizational characteristics, different situations
are related to supportive, constrained, and lack of administrative
conditions for decision making process. For the chain characteris-
tics, low, moderate and high risk situations stand for low, restricted
and high dependence and reliability on other supply chain actors,
respectively (Kirezieva, Nanyunja, et al., 2013).
2.2.2. Diagnosis of performance of control and assurance activities
For each control and assurance activities, present in the FSMS,
indicators are included to measure the performance of the FSMS.
There are four situational descriptions which stand for low (situa-
tion 1), basic (situation 2), average (situation 3), and advanced
(situation 4) level of a particular activity in control or assurance. For
control or assurance activity, situation 1 corresponds to the situa-
tion which is not conducted or applied, or there is no available
information (score 0).
For control activities, situation 2 stands for the usage of own
experience and general knowledge, standard equipment with un-
known capability, ad-hoc analysis, instability and regular unex-
pected problems, and activities which are not written in procedures
(score 1). Situation 3 is related to the usage of expert, specific
knowledge, guidelines designed for specific sector, best practices
and standardized methods with occasional known issues and
problems (score 2). Situation 4 is related to the usage of very spe-
cific information and scientific knowledge, critical analysis, sys-
tematic methodology and procedures (score 3).
Assurance activities were ranked as follows: situation 2 is
typified by problem-driven circumstances, only checked, and rarely
reported (score 1), while situation 3 stands for active translation of
stakeholders requirements, additional analysis and regular docu-
mentation and reporting, with the expert support (score 2). Situ-
ation 4 corresponds to pro-active translation of requirements,
actual observations and testing (score 3) (Kirezieva, Jacxsens, et al.,
2013).
2.2.3. Diagnosis of food safety output
The output of the activities is measured using performance in-
dicators related to sampling, internal evaluation as internal audit-
ing, non conformity assessment, number and type of reported food
safety problem (e.g. complaints) and evaluation of FSMS by external
parties. Four situations are related to no information (situation 1),
poor (situation 2), moderate (situation 3), and good food safety
output (situation 4). Poor output is associated with ad-hoc sam-
pling, minimal criteria used for FSMS evaluation, and existence of
different food safety problems or remarks during inspections and
audits. Moderate output corresponds to regular sampling, several
criteria used for evaluation, and restricted food safety problems,
mainly due to one (restricted) type of problem. Good output rep-
resents systematic evaluation of the FSMS using specific, tailored
criteria without food safety problems or important remarks during
inspections and audits (Jacxsens et al., 2010; Kirezieva, Jacxsens,
et al., 2013).
2.3. Data analysis
Data processing was performed using Microsoft Office Excel.
The scores for context riskiness ranged from 1 (low risk) to 3 (high
risk), while FSMS activity performance and food safety output
scores ranged from 1 (not applied or done) to 4 (advanced for FSMS
activities and good for food safety output). Mode values were
calculated as the most frequent score for a given indicator over the
different producers and cold store companies.
3. Results and discussion
3.1. Orchard farms
3.1.1. Context factors
The riskiness of product, process, organizational and chain
environmental characteristics in nine orchard farms are presented
154 A. Rajkovic et al. / Food Control 80 (2017) 151e161

in Table 2 (context of FSMS). As all farms involved in this study are
producing the same type of product, following the same practices
and cultivation system, the indicators for product and process
characteristics scored similarly for all farms, reflecting moderate to
high risk context. Risk related to microbiological and chemical
contamination of initial (planting) material was moderate (CP1,
CP2, CP3, score 2), while risk related to the final product was high
(CP4, CP5, score 3). During cultivation from buds there is a possi-
bility of fungi development, and although raspberries are rarely
associated with mycotoxin contamination, patulin presence has
been reported at levels of 746 mg/kg (Drusch & Ragab, 2003) and
low levels of alternariol were found in raspberry juices (Lau et al.,
2003). Therefore, all nine orchard farms scored 2 for mycotoxin
production, indicating moderate risk (CP3, Table 2). Additionally,
final fruits are susceptible to microbial contamination, due to its
complex surface characteristics, porosity and lack of protective
surface coating (Lynch et al., 2009), but also to pesticide contami-
nation as no additional step will remove undesirable pesticide
residues, putting higher demands on core control activities in the
raspberries chain. The issues pertinent to pesticide residues should
not be underestimated in the raspberry production chain. Among
128 samples that were analyzed, 66 (51.6%) were found to detect
various pesticide residues (Lozowicka, Kaczynski, Jankowska,
Rutkowska, & Hrynko, 2012).
All orchard farms were open cultivation system (CP6, score 3)
and therefore more difficult to control and more susceptible to
microbial contamination due to their direct contact with the soil
and environment. There is a risk posed by livestock, wild animals
(Suslow et al., 2003), and birds (Langholz & Jay-Russell, 2013). In
line with geographical position of Serbia, climatic conditions are
favourable for growing raspberries, with cool winter, long growing
season, when temperatures slowly warm up in spring, and mod-
erate summer (Veljkovic, Glisic, & Leposavic, 2008). Nevertheless,
these conditions occasionally may favour pathogen survival and
growth, which may be introduced or maintained for prolonged
period of time in the production environment (Guber, Shelton,
Pachepsky, Sadeghi, & Sikora, 2006; Parker, McIntyre, & Noble,
2010) and therefore all orchard farms scored 2 for climate condi-
tions (CP7).
Only two orchard farms were applying an irrigation process
using potable water, while other farms used water only to prepare
solution for agri-technical measures (e.g. pesticide solution, fertil-
ization, etc.). Only one farm (Non Global G.A.P.) used water from
uncontrolled source, which presents a high food safety risk (CP8,
score 3). Pre-harvest application of microbial contaminated solu-
tions onto fresh produce could be an important additional source of
microbial contamination (Guan, Blank, & Holley, 2005; Ng, Fleet, &
Heard, 2005). Jacxsens et al. (2017) demonstrated the potential
effect of contaminated surface water used for preparing pesticide
solution on Norovirus exposure on raspberries.
Risks related to organizational and chain environmental char-
acteristics are presented in Table 2. Eight out of nine orchard farms
lacked competent personnel directly involved in food safety issues
(CO9, score 3). They hired workers with no competence criteria and
do not even provide basic food safety training.
Orchard farms that had implemented and certified Global G.A.P.,
had smaller risks related to organizational riskiness, such as man-
agement commitment and policy in respect to food safety. In all
Global G.A.P. certified farms, this was prepared by produce/retail
organization (CO12, score 2), and was not observed in Non Global
G.A.P. certified farms (CO12, score 3).
The requirements from stakeholders (CC16) are mainly related

Table 2
The distribution of the individual scores for the context factors in Non Global G.A.P. (n ¼ 6), Global G.A.P. certified orchard farms (n ¼ 3) and cold stores (n ¼ 8). (Score 1, 2, and 3
represent respectively low, moderate and high-risk context).

Indicators Orchard farms Modec Orchard farms Modec Cold stores Modec
(Non Global G.A.P.) (Global G.A.P.)

1 2 3 1 2 3 1 2 3

Product characteristics
CP1: Risk of initial materials e microbiological 0 6 0 2 0 3 0 2 0 0 8 3
CP2: Risk of initial materials e pesticide 0 6 0 2 0 3 0 2 0 2 6 3
CP3: Risk of initial product e mycotoxins 0 6 0 2 0 3 0 2 0 0 8 3
CP4: Risk of final product e microbiological 0 0 6 3 0 0 3 3 0 0 8 3
CP5: Risk of final product e pesticides 0 0 6 3 0 0 3 3 0 1 7 3
Process characteristics
CP6: Production systema 0 0 6 3 0 0 3 3 0 0 8 3
CP7: Climate conditions 0 6 0 2 0 3 0 2 8 0 0 1
CP8: Water supply 1 4 1 2 2 1 0 1 7 1 0 1
Organizational characteristics
CO9: Technological staffb 0 0 6 3 0 1 2 3 0 5 3 2
CO10:Variability of workforce 2 4 0 2 2 1 0 1 2 6 0 2
CO11: Operator competence 0 0 6 3 0 0 3 3 0 1 7 3
CO12: Management commitment 0 0 6 3 0 3 0 2 4 4 0 2
CO13: Employee involvement 0 0 6 3 0 0 3 3 0 5 3 2
CO14: Formalization 0 0 6 3 0 3 0 2 4 4 0 2
CO15: Information system 0 0 6 3 0 0 3 3 1 4 3 2
Chain characteristics
CC16: Requirements of stakeholders 0 5 1 2 0 3 0 2 1 4 3 2
CC17: Supplier relationships 0 0 6 3 0 0 3 3 2 5 1 2
CC18: Food safety information exchange 0 0 6 3 0 0 3 3 0 1 7 3
CC19: Conditions of the logistic facilities 0 0 6 3 0 1 2 3 5 3 0 1
CC20: Inspections of food safety authorities 0 0 6 3 0 1 2 3 0 5 3 2
CC21: Variability of suppliers for raw materials 0 6 0 2 0 3 0 2 0 4 4 2
CC22: Specificity of external support 0 0 6 3 0 0 3 3 0 0 8 3
CC23: Specificity food safety legal framework 0 6 0 2 0 3 0 2 1 6 1 2
a
For orchard farms, production system is cultivation system, while for cold stores production system is (full/partial) intervention step.
b
Staff having specific knowledge in food safety.
c
Mode represents the most frequent answer among respondents for given indicator.
 A. Rajkovic et al. / Food Control 80 (2017) 151e161
to Global G.A.P. standard, and three out of nine farms were certified
(F1, F2, F4), as a producer group certified with cold stores (CS1 and
CS2). On the other side, there was no influence of orchard farms on
the producers of pesticides and/or fertilizers (CC17, score 3). The
farms were not able to make a clear specification and put re-
quirements on the producers of pesticides and/or fertilizers, mainly
due to the lack of competence. The system is still immature and
needs more time, developments and investments, before these
chain environmental characteristics become of smaller risk for the
food safety of raspberries.
There was high risk in eight out of nine farms regarding the
logistic conditions (transport vehicles and storage facilities, CC19,
score 3). Transport was performed by tractors or tiller trailers,
without temperature control. Only farm F4 was using closed
transportation vehicle, but without any control of temperature,
humidity, gas, etc. It is however important to note the distance
between the farm and the cold store. Farms F1, F2 and F3 are almost
at the vicinity to cold stores, making it easier to deliver raspberries
shortly after the harvest. Opposed to this option, distance between
orchard farms F7, F8 and F9 and cold store CS3 is quite long
(>10 km). These farms are non-reachable by public roads, making it
difficult to control. The transfer of raspberries to the cold store soon
after the harvest minimizes the opportunity for post harvest
contamination.
Overall, for all farms context risks were mostly moderate to
high, putting a great pressure on control and assurance activities in
the FSMS in order to achieve a good and stable food safety output.
Major differences in context riskiness between Global G.A.P. and
Non Global G.A.P. orchard farms were seen in organizational char-
acteristics. In particular, Global G.A.P. certified farms had lower risk
for indicators such as “variability of workforce” (CO10), “manage-
ment commitment” (CO11) and “formalization” (CO14). The effect
of certification on the organization and chain riskiness was also
previously demonstrated across Europe and in the comparison
between Kenya (with Global G.A.P. certified farms) and Uganda (no
certified farms) (Nanyunja et al., 2015a).
3.1.2. Core control activities
Table 3 shows the frequency distribution of scores and mode
values for indicators representing status of design and actual
operation of control measures. In Non Global G.A.P. farms, a lack of
application (score 1) of core control indicators was determined,
especially in monitoring and operation of control activities
(Table 3), while basic (activity implemented based upon own
knowledge) to average level (activity implemented based on
generic, sector generated information) was seen in Global G.A.P.
farms, still not towards the level of tailored to its own situation, as
could be expected from Global G.A.P. certified organisations.
Although all farms are using crates for collecting raspberries
(PM24, score 1), there was a difference in the application of sani-
tation requirements. Non Global G.A.P. farms operated at low level
(PM27, score 1), while Global G.A.P. certified farms operated at basic
level (PM27, score 2). This is mainly related to activities performed
at cold stores, not at the farm. On one side, CS1 and CS2 dedicated
time and place for cleaning the crates, supplying farms with clean
creates. Additionally, orchard farms F1, F2 and F3 were obliged to
use one layer of foil at the bottom of crates. These foils were
delivered by cold store CS1 itself, without leaving this to the
farmers’ good will. On the other side, Non Global G.A.P. farms often
used dirty, soiled and unclean crates. Although all orchard farms
were legally obliged to follow basic hygienic requirements for the
equipment and facilities that come in contact with the fresh berries
(Serbia, 2010c), they rather rely on the support of cold stores, and
they were convinced that crates’ hygiene was not their
responsibilities.
155
One important control activity that needs special attention is
personal hygiene (PM28). Although all farms were applying stan-
dard requirements regarding clothing and personal hygiene, only
workers at orchard farms F4 were using gloves during harvesting,
while others harvested with bear hands. Some farms possessed
movable toilets and hand washing stations; some of them were
located close to house and used house toilets, while only few farms
do not have toilet facilities or hand washing stations at all, not
allowing workers to wash hands before picking berries (Park et al.,
2013). Poor personal hygiene of workers is a well-known risk factor
for the microbial contamination of fresh produce, during growing
in the fields, harvesting, post-harvest handling and processing (Li,
Keuckelaere, & Uyttendaele, 2015). Surely in the control of Nor-
ovirus contamination and further exposure assessment towards
consumers, personal hygiene was identified as a major risk factor
(Jacxsens et al., 2017). The results of Park et al. (2013) indicated that
spinach contamination was significantly reduced when workers
used hand-washing stations or when the farm provided portable
toilets for workers and trained the workers in how to use them.
Organic fertilizer, mainly manure, was used within all investi-
gated farms (PM32), as farmers often have their own cow/sheep
production and therefore solid manure and slurries, as a waste
product. The benefits of using animal manure are mainly related to
the addition of plant nutrients (nitrogen, phosphorus and potas-
sium) and organic matter (Gagliardi & Karns, 2000). However, an-
imal manures frequently contain enteric pathogens and the usage
of untreated and/or improperly treated manure on the field may
lead to the entrance of pathogens into the food chain and to the
contamination of soil, fresh produce, surface, ground water and
drinking water. Appropriate composting, preparation and applica-
tion at the field is needed to prevent contamination of raspberries
(Nicholson, Groves, & Chambers, 2005). Non Global G.A.P. farms
were handling manure based on the previous experience and
applied it without any clear instructions and recommendations
(PM32, score 1). On contrary, Global G.A.P. farms were using
recommendation and advices given by cold stores (CS1 and CS2)
regarding time lap between the manure application and rasp-
berries harvest, and also regarding the minimum time of manure
storage and ripening before its application in the field (PM32 score
2).
The analysis of pesticides application showed that Global G.A.P.
farms operated at higher level (PM33, score 3) compared to Non
Global G.A.P. farms (PM33, score 2). It is of note that this was once
more related to the activities that arose from cold stores. Each
season, cold stores (CS1 and CS2) have prepared pesticides program
with the list of recommended pesticides, maximum permitted
amount and the time for its application, based on the expert
knowledge and instructions derived from pesticides suppliers. They
also delivered recommended pesticides to orchard farms, to avoid
possibilities of using replacement substances.
All orchard farms that have Global G.A.P. certification have been
already obliged to perform external analysis of pesticides according
to pre-defined sampling plan, and therefore they operated at
moderate and advanced level (MS41 and MS42, score 3 or 4),
compare to Non Global G.A.P. certified farms (MS41 and MS42,
score 1).
3.1.3. Core assessment activities in performance of FSMS
Assurance activities of orchard farms showed some differences
(Table 4). They are aiming at providing evidence that the control
activities in place are in function. The observed variability within
the organisations was mainly related to the fact that some orchard
farms had implemented the Global G.A.P. requirements (F1, F2, F4),
while others had not (F2, F5eF9). Global G.A.P. includes several
assurance requirements including internal self-assessment. As
156 A. Rajkovic et al. / Food Control 80 (2017) 151e161

Table 3
The distribution of the individual scores to compare core control activities for Non Global G.A.P. orchard farms (n ¼ 6), Global G.A.P. certified orchard farms (n ¼ 3) and cold
stores (n ¼ 8). (Score 1, 2, 3, and 4 represent low, basic, average and advanced level).

Indicators Orchard farms Modea Orchard farms Modea Cold stores Modea
(Non Global G.A.P.) (Global G.A.P.)

1 2 3 4 1 2 3 4 1 2 3 4

Preventive measures design

PM24: Hygienic design of equipment and facilities 6 0 0 0 1 2 1 0 0 1 0 3 5 0 3
PM25: Maintenance and calibration program 0 6 0 0 2 0 3 0 0 2 0 0 3 5 4
PM26: Storage facilities 6 0 0 0 1 2 1 0 0 1 0 0 7 1 3
PM27: Sanitation program(s) 5 1 0 0 1 0 3 0 0 2 0 1 5 2 3
PM28: Personal hygiene requirements 0 6 0 0 2 0 3 0 0 2 0 0 4 4 3
PM29: Incoming material control 6 0 0 0 1 1 2 0 0 2 0 2 4 2 3
PM30: Packaging equipment 6 0 0 0 1 3 0 0 0 1 1 4 2 1 2
PM31: Supplier control 6 0 0 0 1 3 0 0 0 1 1 1 3 3 3
PM32: Organic fertilization at the farm 5 1 0 0 1 0 3 0 0 2 - - - - -
PM33: Pesticide program 0 5 1 0 2 0 0 3 0 3 - - - - -
PM34: Water control 5 0 1 0 1 0 1 2 0 3 0 0 8 0 3
PM35: Irrigation method 6 0 0 0 1 1 1 1 0 2 - - - - -
Intervention method design
IM36: Full physical intervention - - - - - - - - - - 8 0 0 0 1
IM37: Partial physical intervention 6 0 0 0 1 3 0 0 0 1 8 0 0 0 1
IM38: Chemical intervention strategies - - - - - - - - - - 8 0 0 0 1
Monitoring system design
MS39: Analysis of CCP/CPs - - - - - - - - - - 0 0 8 0 3
MS40: Standards and tolerances design - - - - - - - - - - 0 0 8 0 3
MS41: Analytical methods to assess pathogens 6 0 0 0 1 0 0 2 1 3 0 0 0 8 4
MS42: Analytical methods to assess pesticides 6 0 0 0 1 0 0 2 1 3 0 0 0 8 4
MS43: Measuring equipment to monitor process/product status - - - - - - - - - - 0 1 6 1 3
MS44: Sampling plan for microbial assessment 6 0 0 0 1 0 0 3 0 3 1 2 5 0 3
MS46: Sampling plan for pesticide assessment 6 0 0 0 1 0 0 3 0 3 1 3 4 0 3
MS46: Corrective actions 6 0 0 0 1 0 0 3 0 3 2 1 3 2 3
Operation of control activities
OC47: Actual availability of procedures 6 0 0 0 1 0 0 3 0 3 0 1 5 2 3
OC48: Actual compliance to procedures 6 0 0 0 1 0 0 3 0 3 0 1 5 2 3
OC49: Actual hygienic performance of equipment and facilities 6 0 0 0 1 3 0 0 0 1 2 0 1 5 4
OC50: Actual storage/cooling capacity 5 1 0 0 1 3 0 0 0 1 0 1 4 3 3
OC51: Actual process capability of full intervention processes - - - - - - - - - - 8 0 0 0 1
OC52: Actual process capability of partial physical intervention 6 0 0 0 1 3 0 0 0 1 8 0 0 0 1
OC53: Actual process capability of packaging 6 0 0 0 1 3 0 0 0 1 1 2 3 2 3
OC54: Actual performance of measuring equipment - - - - - - - - - - 0 0 6 2 3
OC55: Actual performance of analytical equipment 6 0 0 0 1 1 0 1 1 3 0 0 0 8 4
a
Mode represents the most frequent answer among respondents for given indicator.

expected, Non Global G.A.P. orchard farms operated at low level for
all elements of assurance activities (score 1, Table 4), as they were
neither mandatory by the legislation (EC, 2004; Serbia, 2010c), nor
by product specifications set by retailers (Luning et al., 2009).
Certification against Global G.A.P. has been obliged as a licence-
to-trade by retailers of three orchard farms involved in this study,
and consequently results indicated that they performed some
assurance activities at basic or average level (score 2 or 3, Table 4),
such as “translation of stakeholder requirements into own FSMS
requirements” (SR56), “documentation” (SU63) and “record keep-
ing” (SU64). Results showed that even for Global G.A.P. certified
farms, validation and verification activities were conducted at low
level (score 1, Table 4) and have to be improved. Certification re-
quirements were often perceived as a burden to trade, because
setting requirement on a FSMS demands from producers the higher
level of development assurance activities. This was obvious from
the study of Nanyunja et al. (2015a, 2015b), when comparing the
situation of Kenyan certified farmers compared to non certified
farmers in Uganda.
3.1.4. Food safety output
The distribution of scores for the system output indicators are
presented in Table 5. All external and internal management in-
dicators were at low level for Non Global G.A.P. orchard farms
(EM65-69, IM70-74, score 1). This further indicates that these farms
have not been externally evaluated by either competent authority
or third party audits (EM65, score 1). They lacked the registration of
customer complaints of any kind (microbiological, chemical or vi-
sual) and have no insight or knowledge with the quality and safety
of the product they produce and deliver.
On the other side, Global G.A.P. certified orchard farms operated
at moderate to high levels for most of the external FSMS output
indicators (EM65-69, score 3 or 4). The external food safety audits
in Global G.A.P. orchard farms have been carried out by an
accredited certification body (EM65, score 3). It is important to
emphasize, that these certified farms had only minor remarks on
specific aspects of FSMS (EM66, score 4). Although no microbio-
logical or chemical complains have been noted (EM67 and EM68,
score 4), various quality complains occurred in F1 and F2 (EM69,
score 2), which might indicate some problems in the functioning of
the FSMS. Introduction of a customer complaint log and their
further analysis, may give the company information and the di-
rection for the improvements of FSMS. All three Global G.A.P. or-
chard farms performed sampling of final products, only upon
request (IM70 and IM72, score 2). The interpretation of both
microbiological and chemical results was done using both legal
criteria, requirements and specifications set by major stakeholders
and customers (IM71 and IM73, score 2 and 3). Taking samples and
registering non-conformities may allow orchard farms to judge the
performance of their FSMS.
Differences observed in food safety output between Global
G.A.P. and Non Global G.A.P. orchard farms in Serbia, have been also
 A. Rajkovic et al. / Food Control 80 (2017) 151e161 157

Table 4
The distribution of the individual scores to compare assurance activities for Non Global G.A.P. orchard farms (n ¼ 6), Global G.A.P. certified orchard farms (n ¼ 3) and cold stores
(n ¼ 8). (Score 1, 2, 3, and 4 represent low, basic, average and advanced level).

Indicators Orchard farms Modea Orchard farms Modea Cold stores Modea
(Non Global (Global G.A.P.)
G.A.P.)

1 2 3 4 1 2 3 4 1 2 3 4

Setting of system requirements

SR56: Translation of stakeholder requirements into own management 6 0 0 0 1 1 1 1 0 2 2 2 3 1 3
requirements
SR57: The systematic use of feedback information to modify 6 0 0 0 1 2 1 0 0 1 2 2 4 0 3
management system
Validation
VA58: Validation of preventive measures 6 0 0 0 1 3 0 0 0 1 0 3 5 0 3
VA59: Validation of intervention processes 6 0 0 0 1 3 0 0 0 1 1 7 0 0 2
VA60: Validation of monitoring systems - - - - - - - - - - 0 5 3 0 2
Verification
VE61: Verification of people related performance 6 0 0 0 1 3 0 0 0 1 1 3 4 0 3
VE62: Verification of equipment and methods related performance 6 0 0 0 1 3 0 0 0 1 0 5 3 0 2
Support
SU63: Documentation 6 0 0 0 1 0 0 3 0 3 0 2 6 0 3
SU64: Record keeping system 3 3 0 0 2 0 2 1 0 2 0 1 7 0 3
a
Mode represents the most frequent answer among respondents for given indicator.

determined between certified green beans farms in Kenya and non- and final products (CP1, CP3, score 3). Two processors (CS1 and CS2)
certified hot pepper farms in Uganda (Nanyunja et al., 2015a) and operated at a moderate risk with respect to pesticide contamina-
between certified and non-certified mushroom farms in South Af- tion for raw material and final product (CP2, CP5, score 2), as they
rica (Dzingirayi & Korsten, 2016). sourced raw materials from Global G.A.P. certified orchards (F1, F2,
F3 and F4), in which requirements related to proper application of
only approved pesticide have to be fulfilled (Sawe et al., 2014). In
3.2. Cold stores
addition, these cold stores directly delivered recommended pesti-
cides to orchard farms, to avoid usage of found-in-pharmacy pes-
3.2.1. Context factors
ticides. The other six cold stores (CS3-CS8) scored high risk related
The riskiness of product, process, organizational and chain
to pesticide residues (CP2, CP5, score 3).
environmental characteristics in eight cold stores are presented in
Raspberries are very fragile fruits and therefore post harvest
Table 2. The production of frozen raspberries consists of selection,
decontamination treatments, not affecting the texture and nature
freezing and packaging of raspberries, without any physical or
of the produce, are not available. Even a simple washing step, often
chemical intervention step which might reduce the level of
applied in the production of other fresh produce (e.g. leafy green
potentially present microorganisms or chemicals. The microbio-
vegetables), is not suitable, due to tissue damage. Several studies
logical (bacterial and viral) and/or chemical (pesticide residues)
have shown that freezing and frozen storage are actually unable to
contamination of raspberries (Van Boxstael et al., 2013), that occurs
completely remove or inactivate enteric viruses (Baert, Debevere, &
at farms remains in the cold stores. It is of note that companies
Uyttendaele, 2009; Butot, Putallaz, & Sa
nchez, 2008), which are
were aware of this problem and already advised by their consul-
reported to be a major food safety threat for frozen raspberries (Li
tants/certification bodies to introduce incoming control as a “con-
et al., 2015). As a consequence, all eight cold stores operated at high
trol” or even “critical control point”, without any (control)
risk for the production system (CP6, score 3). There was a low risk
mechanism to decrease the risk. Consequently, all cold stores
in all companies regarding climate conditions (CP7, score 1). Due to
operated at high risk microbial contamination for initial materials

Table 5
The distribution of the individual scores to compare food safety performance Non Global G.A.P. orchard farms (n ¼ 6), Global G.A.P. certified orchard farms (n ¼ 3) and cold
stores (n ¼ 8). (Score 1, 2, 3, and 4 represent low, poor, moderate and good food safety performance).

Indicators Orchard farms Modea Orchard farms Modea Cold stores Modea
(Non Global (Global G.A.P.)
G.A.P.)

1 2 3 4 1 2 3 4 1 2 3 4

External management indicators

EM65: FSMS evaluation 6 0 0 0 1 0 0 3 0 3 1 0 2 5 4
EM66: Seriousness of remarks 6 0 0 0 1 0 0 0 3 4 1 0 1 6 4
EM67: Hygiene related and microbiological food safety complaints 6 0 0 0 1 0 0 0 3 4 0 0 2 6 4
EM68: Chemical safety complaints 6 0 0 0 1 0 0 0 3 4 0 0 0 8 4
EM69: (Visual) quality complaints 5 1 0 0 1 0 2 0 1 2 0 2 3 3 3
Internal management indicators
IM70: Product sampling to confirm microbiological performance 6 0 0 0 1 0 3 0 0 2 0 1 4 3 3
IM71:Judgment criteria are used to interpret microbiological results 6 0 0 0 1 0 1 2 0 3 0 1 6 1 3
IM72:Product sampling to confirm use of pesticides 6 0 0 0 1 0 3 0 0 2 0 1 7 0 3
IM73:Judgment criteria are used to interpret pesticide testing results 6 0 0 0 1 0 1 2 0 3 0 1 6 1 3
IM74:Non conformities 6 0 0 0 1 0 0 2 1 3 0 0 2 6 4
a
Mode represents the most frequent answer among respondents for given indicator.
158 A. Rajkovic et al. / Food Control 80 (2017) 151e161
the processing technology, the production of frozen raspberries has
to be performed in well controlled and acclimatized conditions,
that do not favour microbial growth.
Seven cold stores used only potable water in their production
process, and therefore operated at low risk (CP8, score 1). One
company operated at moderate risk (CP8, score 2), due to parallel
usage of potable and surface water, without statistically under-
pinned sampling plan for testing surface water. Although there was
no direct contact between raspberries and potentially contami-
nated water, workers were still using water for hand washing,
cleaning equipment and other contact surfaces. Several studies
have indicated that the usage of inadequate quality water can be
the cause of food safety issues in the fresh produce chain (Gil,
Selma, Lo
pez-Ga

lvez, & Allende, 2009; Hedberg et al., 1999;
Holvoet, Jacxsens, Sampers, & Uyttendaele, 2012).
Regarding the organizational characteristics, some differences
have been noticed among companies. All of them had restricted
number of people dedicated to food safety and quality issues, and
none of the companies had specialists, food microbiologists or food
safety experts. This is probably due to the size of companies (being
mostly medium size) and expertise in this field, that is mainly
obtained from the relevant institutions. Although the companies
had a relative consistent workforce composition (CO10, scores 1
and 2), the absence of adequate operator capability created a high
risk context situation, as the limited number of available workers
had specific knowledge and experience in the area of food safety
(CO11, score 3). In order to obtain consistent food safety practices
during food production, the adequate education, training and
ability to implement obtained knowledge is needed (Djekic et al.,
2014; Jevsnik, Hlebec, & Raspor, 2008; Ko, 2010). Nevertheless,
the companies had relative consistent workforce composition,
which allowed companies to have in-house trainings and routine
performance of food safety procedures. Other organizational
characteristics, such as “management commitment” (CO12),
“employee involvement” (CO13) and “formalization” (CO14) were
at low-moderate risk (scores 1 and 2), and represent places for
further improvements.
It is important to note that the level of information system was
low in most companies (CO14, scores 2 and 3) and consequently no
proper food safety decision making process could be performed
(McMeekin et al., 2006). Additional requirements have to be set for
the verification activities.
The riskiness in chain environment characteristics differs
among eight cold stores, since they have created different relations
with the suppliers of raw material/fresh raspberries and with the
purchasers of frozen raspberries. Regarding suppliers re-
quirements, two companies (CS1 and CS2) had an influence on the
suppliers’ FSMS, and they organized and helped local raspberries
suppliers of raspberries to implement and certify Global G.A.P. at
their farms (CC17, score, 1). Good communication between cold
stores and orchard farms (as major suppliers) allowed better un-
derstanding and problem solving situations, putting less demand
on the control and core assurance activities in FSMS. Nevertheless,
other companies operated mostly at moderate (CC17, score 2) to
high risk (CC17, score 3), indicating minor influence of company on
the suppliers’ performance, which may result in more unpredict-
able safety levels of fresh raspberries.
After the cold store, retail is the next actor in raspberries food
chain. It is important to determine what is the relation between
cold stores and next partner in the chain, and how this relation
influences FSMS of cold stores. Three companies operated at high
risk (CC16, score 3), as their major stakeholders created different
and strict quality assurance requirements on company’s FSMS. This
is mainly related to the fact that they exported frozen berries to big
retailers and different countries (EU countries, USA, Japan), where
different food safety legislation is in force. Other companies had
either generally recognized quality assurance requirements (CC16,
score 1) or additional requirements common to all stakeholders
(such as ISO, SQF, BRS) (CC16, score 2). Strict and various re-
quirements placed on FSMS by stakeholders also put higher de-
mands on FSMS requiring advanced control and assurance
activities.
As the major final product is frozen berries, the logistic facilities
used to transport final products to the next actor in the chain are
nonnegotiable, and they need strict control of environmental
conditions. For this indicator, five companies operated at low risk
(CC19, score 1), while three companies operated at moderate risk
level (CC19, score 2).
The high level of riskiness was given to characteristics such as
“Inspections of food safety authorities” (CC20), as inspection was
unsystematic and unspecific, without adequate follow-up activates
and no feedback information was ever given to the companies, as
well as “specificity of external control” (CC22), due to the lack of
specific product or production system external support.
3.2.2. Core control activities
The scores and frequencies for core control activities in eight
companies are presented in Table 3. All processors placed personnel
hygiene requirements at a moderate (PM28, score 3) or advanced
level (PM28, score 4). This means that they have placed high re-
quirements for all food handlers on clothing, personal care and
health, specific training and hygiene instructions. Other preventive
control measures such as “hygienic design of equipment and stor-
age facilities” (PM24), proper usage of cleaning agents within
“sanitation programs” (PM27), were mainly at the moderate or
advanced level (score 3 or 4) in examined food businesses. This was
mainly connected with the hygiene regulation, which puts strict
demands on good manufacturing practices and control activities
(Serbia, 2010c). Preventive measures influence food safety by
reducing the chances on (cross) contamination or microbial
growth. A full-step and tailored sanitation program with appro-
priate cleaning agents, prevents contamination, and five out of
eight companies indicated the usage of complete sanitation pro-
gram with cleaning agents advised by suppliers (PM27, score 3),
and only two companies showed advanced level of sanitation
program which verified effectiveness on-site (PM27, score 4). This is
of great importance, having in mind the fact that disinfection and
cleaning may not be always effective, and persistent pathogens may
occur after the sanitation procedures take place in the production
area (The
venot et al., 2006).
There was low application (score 1) of two preventive in-
dicators, one for “packaging equipment” (PM30), due to the lack of
packaging machine and manual manipulation during packaging,
and the other for total absence of “supplier control” in one cold
store (PM31).
All companies assigned low level control activities to either full
or partial post-harvest processing (IM36-37, score 1), as they were
not used in cold stores (Richards, 2001). Partial intervention is used
to reduce pathogen load on fresh produce, and this includes
washing, disinfection, removal of outer layers, etc. On the other side
full intervention processes include heat treatment such as
blanching, pasteurization, sterilization and drying (Kirezieva et al.,
2015). Nevertheless, neither full nor partial intervention was
applied for raspberries, due to their fragile nature.
All companies had an implemented HACCP, as it became a
mandatory legal requirement since 2009 (Serbia, 2009). Therefore
all of them performed hazard identification, risk assessment and
allocation of critical control points/control points based on hygiene
codes and this was performed by external experts/consultants. The
companies were not aware of the existence and information about
 A. Rajkovic et al. / Food Control 80 (2017) 151e161
actual pathogen behaviour, that can be predicted by using chal-
lenge studies, predictive models, etc. (Augustin et al., 2011; CAC,
2008, pp. 69e2008; Stringer, 2005; Uyttendaele et al., 2004).
The tolerances of product/process parameters and standard
design derived from general hygiene codes (MS40, score 3). Addi-
tionally, analytical test methods for both pesticide residues and
pathogens were internationally validated methods (ISO methods)
performed in the external laboratories, and therefore all companies
scored 4 (MS41 and MS42). The reason for this is recent harmoni-
zation of legal rules with EU legislation (Serbia, 2010a, 2010b), and
requirements for the usage of internationally validated methods
(e.g. ISO methods) instead of methods earlier published in national
Ordinances (Yugoslavia, 1980). Although detailed sampling plan for
microbiological analysis is available within the current legal
document (Serbia, 2010a), the companies varied in scoring factor
“Sampling plan for microbial assessment” (MS44). Five companies
scored 3, indicating the usage of acknowledged guidelines infor-
mation for sampling plans, most probably given within the Ordi-
nance (Serbia, 2010a). However, two companies scored 2 and one
scored 1 for this indicator, signifying that sampling plans used in
these companies were based solely on in-house knowledge and
previous experience or were completely omitted and ad-hoc
samples were sent to laboratory for the analysis. At the same
time, all companies outsourced the laboratory for analysis of
samples, no matter how they were sampled. At the moment of
performing the interviews, no legal documents were published
regarding sampling procedures and instructions for food sampling
in respect to pesticide residues and therefore only four companies
were using common sampling plans for the sector.
The existence of accurate and understandable procedures at the
right place allows a better direction of peoples’ decision-making
behaviour in control activities and all companies scored 3 or 4,
being at high or advanced level (OC47). Along with the existence,
compliance of procedures was also at high level (OC48), probably
due to simple steps in the production. Although, the procedures
were in place and HACCP system was implemented, two companies
lack information about hygienic performance of their equipment
(OC49, score 1). Nevertheless, five companies indicated that the
hygienic performance of their equipment and facilities was stable,
according to the results of regular hygienic performance tests
(OC49, score 4).
Due to specific production of raspberries and requirements for
strict temperature control, influencing both microbial and quality
stability of products, most companies operated at high/advanced
level for the cooling/storage capacity (OC50, score 3 and 4). Envi-
ronmental and product conditions (temperature, humidity) were
automatically monitored and results were analyzed on a regular
basis.
3.2.3. Core assurance activities
The core assurance activities in examined cold stores were
mainly at medium or high level (Table 4). The results for assurance
activities, such as “transfer of stakeholder requirements into own
FSMS” varied from low to advanced level (SR56). The reason for this
variability is most probably the nature of businesses, whether the
final product is distributed on export market (EU, USA or Japan) or
mainly on local market. Mostly, the companies which cooperate
and export their products were required to implement different QA
guidelines and standards such as HACCP, BRC, FSSC 22000 prior of
signing a contract. This further means that assurance activities such
as validation and verification of implemented control measures
were mandatory elements. Nevertheless, some of the companies
were working for local market or distributors whose requirements
were not strict in sense of food safety, but more focused on quality
parameters. The translation of stakeholders requirements is very
159
important when designing and implementing an effective FSMS.
This process accounts for comparison of current control and
assurance activities to the (new) stakeholder requirements, and as
a result a new specific system requirements may be determined
(new sampling plans to asses pathogens or pesticides, more accu-
rate equipment, etc.) (Luning et al., 2009). Additionally, all infor-
mation obtained from feedback may serve to update and modify
FSMS.
Validation of preventive measures and monitoring system is
mainly based on historical data and available knowledge or regu-
latory documents (such as specific hygiene codes), and usually
performed by own company’s people or in some cases by external
experts (CAC, 2008, pp. 69e2008). A more scientific evidence-
based, systematic, and independent validation would improve
both the preventive and monitoring activities. Verification of em-
ployees’ performance in cold stores was based on either checking
the presence and availability of needed procedures and records or
on analyzing procedures (both content and presence) and records.
It is obvious from obtained results that these assurance activities
could be improved as validation, verification, sampling plans are
important to assure food safety of the final product. Companies
usually found their implementation difficult and time-consuming.
3.2.4. Food safety output
As already mentioned, several cold stores were mainly export
oriented, working with different customers all over the world, and
therefore they were often audited from several different parties
(second and third party audits). Two companies were always
audited by one accredited party, and in one company audit of FSMS
was never performed (Table 5). In most audited companies (six out
of eight), no major nonconformities and/or only minor remarks on
specific or various aspects of the FSMS were raised. Additionally,
companies indicated that complains were mainly related to the
quality characteristics, more often than microbiological or chemical
issues. This further indicated a good performance of food safety in
cold stores involved in this study.
4. Conclusion
Due to fragile nature of raspberries, only minimal processing
was undertaken post-harvest, such as freezing, packing, labelling
and distribution. As there was no processing step that will elimi-
nate/reduce food safety hazards (microbiological or chemical) to an
acceptable level, it is important to adequately address these haz-
ards pre-harvest and during harvesting (Manning & Mei Soon,
2013). Therefore the implementation of a private, risk based stan-
dard at the primary production level, such as Global G.A.P. might
serve as a solid foundation to deal with major recognized hazards at
primary production, but also for downstream actors in the rasp-
berries chain, such as processing companies and retail. Although
there was a high risk for product and process characteristics in all
farms involved in this study, for organizational characteristics
smaller context risk was determined for Global G.A.P. certified
farms. As expected certified farms performed higher level of control
activities compared to Non Global G.A.P. farms, as most of control
activities were actually requirements for Global G.A.P. Conse-
quently, food safety output for Global G.A.P. farms was at higher
level compared to Non Global G.A.P. farms. The obtained results
were related not only to the activities at the farms, but also to the
support and help from cold stores. Our results indicated better food
safety output in cold stores compared to orchard farms, as they
operated at higher level of core control and assurance activities.
This highlights several places for food safety improvements
along the raspberries chain. At the primary production level, in-
crease of awareness of risks originating at the farm level is of
160 A. Rajkovic et al. / Food Control 80 (2017) 151e161
ultimate importance. Creating a “food safety culture” including
enhancing risk awareness, communication, leadership, resources
and commitment has been recently identified to be of utmost
importance, even in small scale organisations (e.g. De Boeck,
Jacxsens, Bollaerts, Uyttendaele, and Vlerick (2016)) In addition,
improvements can be connected with water control, pesticides
program, hygiene requirements, storage facilities. At the level of
processing (cold stores), hygienic design and equipment, as well as
the storage capacity should be tested for the specific company and
their own conditions. In addition, validation and verification of
control activities could be improved, as currently they are at
moderate or basic level.
Acknowledgments
This research was conducted within the framework of the Eu-
ropean Union Framework Programme 7 Veg-i-Trade project
“Impact of Climate Change and Globalization on Safety of Fresh
Produce Governing a Supply Chain of Uncompromised Food Sov-
ereignty” (www.veg-i-trade.org, Grant Agreement No. 244994).
References
Augustin, J.-C., Bergis, H., Midelet-Bourdin, G., Cornu, M., Couvert, O., Denis, C., et al.
(2011). Design of challenge testing experiments to assess the variability of
Listeria monocytogenes growth in foods. Food Microbiology, 28(4), 746e754.
Baert, L., Debevere, J., & Uyttendaele, M. (2009). The efficacy of preservation
methods to inactivate foodborne viruses. International Journal of Food Microbi-
ology, 131(2e3), 83e94.
Berger, C. N., Sodha, S. V., Shaw, R. K., Griffin, P. M., Pink, D., Hand, P., et al. (2010).
Fresh fruit and vegetables as vehicles for the transmission of human pathogens.
Environmental Microbiology, 12(9), 2385e2397.
Bouwknegt, M., Verhaelen, K., Rzezutka, _ A., Kozyra, I., Maunula, L., von
Bonsdorff, C.-H., et al. (2015). Quantitative farm-to-fork risk assessment model
for norovirus and hepatitis A virus in European leafy green vegetable and berry
fruit supply chains. International Journal of Food Microbiology, 198, 50e58.
Buric, T. (2003). Foreign agricultural service/USDA, GAIN report. Global Agriculture
Information Network, Yugoslavia Fresh Deciduous Fruit (Raspberries Annual,
Voluntary Report).

nchez, G. (2008). Effects of sanitation, freezing and frozen
Butot, S., Putallaz, T., & Sa
storage on enteric viruses in berries and herbs. International Journal of Food
Microbiology, 126(1e2), 30e35.
CAC. (2008). Guidelines for the validation of food safety control measures. CAC/GL.
Caceres, V. M., Ball, R. T., Somerfeldt, S. A., Mackey, R. L., Nichols, S. E.,
MacKenzie, W. R., et al. (1998). A foodborne outbreak of cyclosporiasis caused
by imported raspberries. Journal of Family Practice, 47(3), 231e235.
Chigor, V. N., Umoh, V. J., Okuofu, C. A., Ameh, J. B., Igbinosa, E. O., & Okoh, A. I.
(2012). Water quality assessment: Surface water sources used for drinking and
irrigation in Zaria, Nigeria are a public health hazard. Environmental Monitoring
and Assessment, 184(5), 3389e3400.
Cotterelle, B., Drougard, C., Rolland, J., Becamel, M., Boudon, M., Pinede, S., et al.
(2005). Outbreak of norovirus infection associated with the consumption of
frozen raspberries, France, March 2005. Eurosurveillance, 10(17), 2690.
De Boeck, E., Jacxsens, L., Bollaerts, M., Uyttendaele, M., & Vlerick, P. (2016). Inter-
play between food safety climate, food safety management system and
microbiological hygiene in farm butcheries and affiliated butcher shops. Food
Control, 65, 78e91.
Djekic, I., Smigic, N., Kalogianni, E. P., Rocha, A., Zamioudi, L., & Pacheco, R. (2014).
Food hygiene practices in different food establishments. Food Control, 39,
34e40.
Drusch, S., & Ragab, W. (2003). Mycotoxins in fruits, fruit juices, and dried fruits.
Journal of Food Protection, 66(8), 1514e1527.
Dzingirayi, G., & Korsten, L. (2016). Assessment of primary production of Horti-
cultural safety management systems of mushroom farms in South Africa.
Journal of Food Protection, 79(7), 1188e1196.
EC. (2004). Regulation (EC) No 852/2004 of the European parliament and of the council
of 29 April 2004 on the hygiene of foodstuffs.
Forslund, A., Ensink, J., Battilani, A., Kljujev, I., Gola, S., Raicevic, V., et al. (2010).
Faecal contamination and hygiene aspect associated with the use of treated
wastewater and canal water for irrigation of potatoes (Solanum tuberosum).
Agricultural Water Management, 98(3), 440e450.
Gagliardi, J. V., & Karns, J. S. (2000). Leaching of Escherichia coli O157: H7 in diverse
soils under various agricultural management practices. Applied and Environ-
mental Microbiology, 66(3), 877e883.
Gibbs, R., Nanyonjo, R., Pingault, N., Combs, B., Mazzucchelli, T., Armstrong, P., et al.
(2013). An outbreak of Cyclospora infection on a cruise ship. Epidemiology and
Infection, 141(3), 508e516.
Gil, M. I., Selma, M. V., Lo
pez-Ga
lvez, F., & Allende, A. (2009). Fresh-cut product
sanitation and wash water disinfection: Problems and solutions. International
Journal of Food Microbiology, 134(1e2), 37e45.
Guan, T. T. Y., Blank, G., & Holley, R. A. (2005). Survival of pathogenic bacteria in
pesticide solutions and on treated tomato plants. Journal of Food Protection,
68(2), 296e304.
Guber, A., Shelton, D., Pachepsky, Y., Sadeghi, A., & Sikora, L. (2006). Rainfall-
induced release of fecal coliforms and other manure constituents: Comparison
and modeling. Applied and Environmental Microbiology, 72(12), 7531e7539.
Hedberg, C., Angulo, F., White, K., Langkop, C., Schell, W., Stobierski, M., et al. (1999).
Outbreaks of salmonellosis associated with eating uncooked tomatoes: Impli-
cations for public health. Epidemiology and Infection, 122(03), 385e393.
Herwaldt, B. L., & Ackers, M.-L. (1997). An outbreak in 1996 of cyclosporiasis
associated with imported raspberries. New England Journal of Medicine, 336(22),
1548e1556.
Holvoet, K., Jacxsens, L., Sampers, I., & Uyttendaele, M. (2012). Insight into the
prevalence and distribution of microbial contamination to evaluate water
management in the fresh produce processing industry. Journal of Food Protec-
tion, 75(4), 671e681.
Jacxsens, L., Stals, A., De Keuckelaere, A., Deliens, B., Rajkovic, A., & Uyttendaele, M.
(2017). Quantitative farm-to-fork human norovirus exposure assessment of
individually quick frozen raspberries and raspberry puree. International Journal
of Food Microbiology, 242, 87e97.
Jacxsens, L., Uyttendaele, M., Devlieghere, F., Rovira, J., Gomez, S. O., & Luning, P.
(2010). Food safety performance indicators to benchmark food safety output of
food safety management systems. International Journal of Food Microbiology, 141,
S180eS187.
Jevsnik, M., Hlebec, V., & Raspor, P. (2008). Food safety knowledge and practices
among food handlers in Slovenia. Food Control, 19(12), 1107e1118.
Kirezieva, K., Jacxsens, L., Uyttendaele, M., Van Boekel, M. A. J. S., & Luning, P. A.
(2013). Assessment of food safety management systems in the global fresh
produce chain. Food Research International, 52(1), 230e242.
Kirezieva, K., Luning, P. A., Jacxsens, L., Allende, A., Johannessen, G. S., Tondo, E. C.,
et al. (2015). Factors affecting the status of food safety management systems in
the global fresh produce chain. Food Control, 52, 85e97.
Kirezieva, K., Nanyunja, J., Jacxsens, L., van der Vorst, J. G. A. J., Uyttendaele, M., &
Luning, P. A. (2013). Context factors affecting design and operation of food
safety management systems in the fresh produce chain. Trends in Food Science &
Technology, 32(2), 108e127.
Ko, W.-H. (2010). Evaluating food safety perceptions and practices for agricultural
food handler. Food Control, 21(4), 450e455.
Langholz, J. A., & Jay-Russell, M. T. (2013). Potential role of wildlife in pathogenic
contamination of fresh produce. HumaneWildlife Interactions, 7(1), 140e157.
Lau, B. P. Y., Scott, P. M., Lewis, D. A., Kanhere, S. R., Cle
roux, C., & Roscoe, V. A.
(2003). Liquid chromatographyemass spectrometry and liquid chromatogra-
phyetandem mass spectrometry of the Alternaria mycotoxins alternariol and
alternariol monomethyl ether in fruit juices and beverages. Journal of Chro-
matography A, 998(1e2), 119e131.
Leposavic, A., Jankovi, M., Ðurovi, D., Veljkovi, B., Keserovi, Z., Popovi, B., et al.
(2013). Fruit quality of red raspberry cultivars and selections grown in Western
Serbia. Horticultural Science, 40(4), 154e161.
Li, D., Keuckelaere, A., & Uyttendaele, M. (2015). Fate of foodborne viruses in the
“Farm to Fork” chain of fresh produce. Comprehensive Reviews in Food Science
and Food Safety, 14(6), 755e770.
Lozowicka, B., Kaczynski, P., Jankowska, M., Rutkowska, E., & Hrynko, I. (2012).
Pesticide residues in raspberries (Rubus idaeus L.) and dietary risk assessment.
Food Additives & Contaminants Part B-Surveillance, 5(3), 165e171.
Luning, P. A., Bango, L., Kussaga, J., Rovira, J., & Marcelis, W. J. (2008). Comprehensive
analysis and differentiated assessment of food safety control systems: A diag-
nostic instrument. Trends in Food Science & Technology, 19(10), 522e534.
Luning, P. A., Marcelis, W. J., Rovira, J., Van Boekel, M. A. J. S., Uyttendaele, M., &
Jacxsens, L. (2011). A tool to diagnose context riskiness in view of food safety
activities and microbiological safety output. Trends in Food Science & Technology,
22, S67eS79.
Luning, P. A., Marcelis, W. J., Rovira, J., Van der Spiegel, M., Uyttendaele, M., &
Jacxsens, L. (2009). Systematic assessment of core assurance activities in a
company specific food safety management system. Trends in Food Science &
Technology, 20(6e7), 300e312.
Lynch, M. F., Tauxe, R. V., & Hedberg, C. W. (2009). The growing burden of foodborne
outbreaks due to contaminated fresh produce: Risks and opportunities. Epide-
miology & Infection, 137(3), 307e315.
Manning, L., & Mei Soon, J. (2013). GAP framework for fresh produce supply. British
Food Journal, 115(6), 796e820.
McMeekin, T. A., Baranyi, J., Bowman, J., Dalgaard, P., Kirk, M., Ross, T., et al. (2006).
Information systems in food safety management. International Journal of Food
Microbiology, 112(3), 181e194.
Nanyunja, J., Jacxsens, L., Kirezieva, K., Kaaya, A. N., Uyttendaele, M., & Luning, P. A.
(2015a). Assessing the status of food safety management systems for fresh
produce production in East Africa: Evidence from certified green bean farms in
Kenya and noncertified hot pepper farms in Uganda. Journal of Food Protection,
78(6), 1081e1089.
Nanyunja, J., Jacxsens, L., Kirezieva, K., Kaaya, A. N., Uyttendaele, M., & Luning, P. A.
(2015b). Shift in performance of food safety management systems in supply
chains: Case of green bean chain in Kenya versus hot pepper chain in Uganda.
Journal of the Science of Food and Agriculture, 96(10), 3380e3392.
Ng, P. J., Fleet, G. H., & Heard, G. M. (2005). Pesticides as a source of microbial
 A. Rajkovic et al. / Food Control 80 (2017) 151e161
contamination of salad vegetables. International Journal of Food Microbiology,
101(2), 237e250.
Nicholson, F. A., Groves, S. J., & Chambers, B. J. (2005). Pathogen survival during
livestock manure storage and following land application. Bioresource Technol-
ogy, 96(2), 135e143.
Nikolic, M., Ivanovic, M., Milenkovic, S., Milivojevic, J., & Milutinovic, M. (2008).
State and prospects of raspberry production in Serbia. Acta Horticulturae, 777,
243e250.
Nikolic, M., & Milivojevic, J. (2015). Jagodaste vo
cke - tehnologija gajenja (in Serbian)
(2nd ed.). Belgrade, Serbia: Univerzitet u Beogradu - Poljoprivredni fakultet.
Ose
s, S. M., Luning, P. A., Jacxsens, L., Santillana, S., Jaime, I., & Rovira, J. (2012). Food
safety management system performance in the lamb chain. Food Control, 25(2),
493e500.
Parker, J., McIntyre, D., & Noble, R. (2010). Characterizing fecal contamination in
stormwater runoff in coastal North Carolina, USA. Water Research, 44(14),
4186e4194.
Park, S., Navratil, S., Gregory, A., Bauer, A., Srinath, I., Jun, M., et al. (2013). Generic
Escherichia coli contamination of spinach at the preharvest level: The role of
farm management and environmental factors. Applied and Environmental
Microbiology, 79(14), 4347e4358.
de Quadros Rodrigues, R., Loiko, M. R., Mine
ia Daniel de Paula, C., Hessel, C. T.,
Jacxsens, L., Uyttendaele, M., et al. (2014). Microbiological contamination linked
to implementation of good agricultural practices in the production of organic
lettuce in Southern Brazil. Food Control, 42, 152e164.
RASFF. (2016). Rapid alert system for food and feed - portal.
Richards, G. (2001). Enteric virus contamination of foods through industrial prac-
tices: A primer on intervention strategies. Journal of Industrial Microbiology and
Biotechnology, 27(2), 117e125.
Sampers, I., Jacxsens, L., Luning, P. A., Marcelis, W. J., Dumoulin, A., &
Uyttendaele, M. (2010). Performance of food safety management systems in
poultry meat preparation processing plants in relation to Campylobacter spp.
contamination. Journal of Food Protection, 73(8), 1447e1457.
Sampers, I., Toyofuku, H., Luning, P. A., Uyttendaele, M., & Jacxsens, L. (2012). Semi-
quantitative study to evaluate the performance of a HACCP-based food safety
management system in Japanese milk processing plants. Food Control, 23(1),
227e233.
Sarvikivi, E., Roivainen, M., Maunula, L., Niskanen, T., Korhonen, T., Lappalainen, M.,
et al. (2012). Multiple norovirus outbreaks linked to imported frozen rasp-
berries. Epidemiology and Infection, 140(2), 260e267.
Sawe, C. T., Onyango, C. M., & Njage, P. M. K. (2014). Current food safety manage-
ment systems in fresh produce exporting industry are associated with lower
performance due to context riskiness: Case study. Food Control, 40, 335e343.
Serbia. (2009). Food safety law. Official Gazette of RS, No. 41/09.
161
Serbia. (2010a). Ordinance on general and specific food hygiene requirements at any
stage of production, processing and trade. Official Gazzette RS, No. 72/10.
Serbia. (2010b). Ordinance on maximum permitted levels of residues of pesticides in
food and feed and feed and feed for which the maximum allowable amount of
residues of plant protection products. Official Gazzette RS, 25/10, 28/11 and 20/13.
Serbia. (2010c). Ordinance on the general conditions of food hygiene. Official Gazette
RS, No 73/10.
Serbia. (2015). Statistical yearbook of the republic of Serbia. Belgrade, Serbia: Sta-
tistical Office of the Republic of Serbia (in Serbian).
Serbia. (2016). Statistical yearbook of the republic of Serbia. Belgrade, Serbia: Sta-
tistical Office of the Republic of Serbia (in Serbian).
Stine, S. W., Song, I., Choi, C. Y., & Gerba, C. P. (2011). Application of pesticide sprays
to fresh produce: A risk assessment for hepatitis A and Salmonella. Food and
Environmental Virology, 3(2), 86e91.
Stringer, M. (2005). Summary report: Food safety objectivesdrole in microbiolog-
ical food safety management. Food Control, 16(9), 775e794.
Suslow, T. V., Oria, M. P., Beuchat, L. R., Garrett, E. H., Parish, M. E., Harris, L. J., et al.
(2003). Production practices as risk factors in microbial food safety of fresh and
fresh-cut produce. Comprehensive Reviews in Food Science and Food Safety, 2,
38e77.
The
venot, D., Delignette-Muller, M. L., Christieans, S., Leroy, S., Kodjo, A., & Vernozy-
Rozand, C. (2006). Serological and molecular ecology of Listeria monocytogenes
isolates collected from 13 French pork meat saltingecuring plants and their
products. International Journal of Food Microbiology, 112(2), 153e161.
USAID. (2008). Cultivated berry value chain assessment - USAID agribusiness project -
Serbia.
Uyttendaele, M., Rajkovic, A., Benos, G., François, K., Devlieghere, F., & Debevere, J.
(2004). Evaluation of a challenge testing protocol to assess the stability of
ready-to-eat cooked meat products against growth of Listeria monocytogenes.
International Journal of Food Microbiology, 90(2), 219e236.
Van Boxstael, S., Habib, I., Jacxsens, L., De Vocht, M., Baert, L., Van De Perre, E., et al.
(2013). Food safety issues in fresh produce: Bacterial pathogens, viruses and
pesticide residues indicated as major concerns by stakeholders in the fresh
produce chain. Food Control, 32(1), 190e197.
Veljkovic, B., Glisic, I., & Leposavic, A. (2008). An analysis of raspberry production
conditions in Serbia. Acta Agriculturae Serbica, 13(25), 9e16.
Verhaelen, K., Bouwknegt, M., Carratala , A., Lodder-Verschoor, F., Diez-Valcarce, M.,
Rodríguez-La
zaro, D., et al. (2013). Virus transfer proportions between gloved
fingertips, soft berries, and lettuce, and associated health risks. International
Journal of Food Microbiology, 166(3), 419e425.
Yugoslavia. (1980). Ordinance on the methods of performing microbiological analyzes
and super analyzes of food. Official Gazzette of SFRY. No. 25/80.