See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/338701749

Time and generational changes in cognitive performance in Romania

Article  in  Intelligence · January 2020

DOI: 10.1016/j.intell.2020.101430

CITATIONS READS
0 710

2 authors:

George Gunnesch-Luca Dragos Iliescu

Friedrich-Alexander-University of Erlangen-Nürnberg University of Bucharest
8 PUBLICATIONS   30 CITATIONS    64 PUBLICATIONS   488 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Modelarea competentelor personalului militar View project

European Journal of Psychological Assessment View project

All content following this page was uploaded by George Gunnesch-Luca on 15 March 2020.

The user has requested enhancement of the downloaded file.

Running head: FLYNN EFFECT IN ROMANIA

Time and Generational Changes in Cognitive Performance in Romania

George Gunnesch-Luca1* & Dragoș Iliescu2*

1
Friedrich-Alexander University Erlangen-Nürnberg
2
University of Bucharest

This version of the article is the preprint version and does not completely replicate the final
authoritative version published in Intelligence at https://doi.org/10.1016/j.intell.2020.101430. It
is not the version of record and is therefore not suitable for citation. Please do not copy or cite
without the permission of the author(s).

Please cite as:

Gunnesch-Luca, G., & Iliescu, D. (2020). Time and generational changes in cognitive
performance in Romania. Intelligence, 79, 101430. https://doi.org/10.1016/j.intell.2020.101430

Author Note

George Gunnesch-Luca is at the School of Business, Economics and Society, FAU

Erlangen-Nuremberg.

Dragoș Iliescu is at the Department of Psychology, University of Bucharest.

*The authors contributed equally to the work.

Declarations of interest: none

Correspondence concerning this article should be addressed to George Gunnesch-Luca, Lange

Gasse 20, 90402, Nuremberg, Germany. E-mail: george.gunnesch-luca@fau.de

FLYNN EFFECT IN ROMANIA 2

Time and Generational Changes in Cognitive Performance in Romania

Few phenomena in psychology have achieved such notoriety as the Flynn Effect (FE).

The FE describes a sustained population increase in cognitive performance that has held up for

much of the 20th century. The observed change varies among the different sampled groups, but it

is generally considered steady, with a ∆IQ/decade ranging from 1.65 in Estonia to an impressive

7.7 in Japan (Williams, 2013). What causes such a drive is still a matter of debate, however

intelligence literature has pinpointed a few variables that might play a role, e.g., access to better

education (Rönnlund & Nilsson, 2008), a decrease in family size (Bjerkedal et al., 2007), or

simply better test-taking strategies (Must & Must, 2013). However, about a decade ago the

picture began to become less clear. In a review of the FE, Williams (2013) noted that it is “… so

specific that for every finding, there seems to be an opposite finding” (p. 755). This assertion is

supported by recent conflicting evidence regarding the FE. For example, Scandinavian countries

started reporting declines in mental test scores (e.g., Dutton & Lynn, 2013; Sundet et al., 2004)

with similar trends being observed in France (Dutton & Lynn, 2015) or Australia (Cotton et al.,

2005). At the same time, the United States or the United Kingdom still enjoy increases in IQ

scores (e.g., Weiss et al., 2016), whereas meta-analytical procedures on German speaking

samples yielded mixed results, reporting increases in vocabulary scores with parallel decreases in

spatial performance (Pietschnig et al., 2010).

In contrast to the continued information stream from developed countries, data that

reflects changes in cognitive performance originating from Eastern-European countries has been

rather scarce. There are many contributing factors as to why, however we believe that causes that

relate to the regional historical context also play an important role. Eastern European communist

regimes have had a disrupting role in the development of psychology as a science, some of them
FLYNN EFFECT IN ROMANIA 3

even breaking the continuity of psychological research in several countries (e.g., Hungary or

Romania). This study contributes to the FE literature by reporting Romanian IQ results from a

pool of yearly repeated measurements (2003-2018). We expand FE knowledge by providing

evidence towards a positive FE in Romania and by analyzing linked generational and period

effects. To this end, we report results from various methodological solutions, that include

bivariate statistics and the more specialized Hierarchical Age-Periods-Cohort (HAPC) models to

improve our understanding of the FE in Romania.

The case for generational variance in Romanian IQ data

A generational cohort has been defined as “a group of individuals similar in age who

have experienced the same historical events within the same time period” (Kowske et al., 2010,

p. 266). Individuals within a generational cohort experience external changes at the same age

simultaneously, while making life decisions concurrently. The members of the same generation

are born, socialize, enter the labor-market and retire approximately together. Members of such a

generational group are also influenced together by external forces, i.e., factors that shape its

characteristics. Research has shown that generations differ in personality traits (e.g., Twenge &

Campbell, 2001), work values (Smola & Sutton, 2002) or attitudes (Kowske et al., 2010).

Romanian society has gone through several major upheavals in the past several decades:

it has experienced the end of the second World War, the radical changes induced by the

communist regimes that followed, their downfall in the 1989 Romanian Revolution and with it,

the re-introduction of a free, democratic society. Such social and political shifts are a magnitude

different from the sometimes linear societal developments in Western societies and, we argue,

are bound to have left residual evidence on populational cognitive performance. We see several

events that might have caused inter-generational differences in IQ. For example, before 1967 the
FLYNN EFFECT IN ROMANIA 4

Romanian policy on abortions was considered extremely liberal. However, the birth rates at that

point were already declining, due perhaps to the ever-increasing participation of women in the

labor market (Dragolea, 2007). This was perceived as both a chance and a threat by the

Communist Party, which, in November 1966, introduced an outright ban on abortions (“The

Decree”) coupled with massive investments in the educational system. One of the consequences

was a doubling of the fertility rate that continued only until approximately 1977, when it returned

to the pre-1967 mean, as Romanian women found clandestine ways to circumvent legislation.

Whatever the goal was, the ban created a massive populational imbalance. An estimated two

million children were born under these conditions, bringing with them high associated costs at

both the individual and societal level: from unwanted children, to supplemental schooling and

medical costs. This generation was later to be known as the “Decree” generation.

Yet another example involves the evolution of literacy rates in Romania during

communism. Totalitarian regimes, by their nature, achieve some objectives faster as they pool

resources and force behaviors towards a goal - and achieving high literacy rates was one of the

explicitly declared goals. Therefore, one positive societal effect was that for the first time in

Romanian history the literacy rates jumped over 90%. This was perhaps due to several factors

among which a major role was played by the increased financial investments made by the

communist regime in the educational system, coupled with a number of educational reforms. As

a consequence, by 1973 almost 91% of the children aged 6-16 were enrolled in one of 2465

schools that operated in Romania (Tudosoiu, 2010). Taken together, these are factors that, we

argue, could have influenced the development of cognitive performance or at least to have

produced inter-generational differences.

FLYNN EFFECT IN ROMANIA 5

The Current Study

This study provides information beyond the simple temporal evolution of cognitive

performance within our sample. Due to the nature of the data, we are also able to deliver

empirical estimates of the long-term effects of generational membership on cognitive

performance, as well as to simultaneously examine the independent effects of age and

measurement period on our variable of interest. We hereby answer the call made by Rodgers

(1998), contributing to the clarification of some issues that we feel are still open, such as the

amount of variance attributable to generational cohorts, and the presence or absence of FE (and

its extent) in less developed countries.

Specifically, this study has three contributions to the literature. First, we report on data

collected in Romania, an understudied Eastern-European country. Second, we report on a large

data set comprising sixteen repeated cross-sectional datasets, collected over a 15-years span,

pooled into a larger pseudo-panel. Third, we analyze the data based on a method (hierarchical

age-period-cohort, or HAPC, analysis) that is uniquely qualified to analyze data in which the

effects of age, period and generation (cohort) are entangled, but that has not been previously

used in the analysis of the FE, in spite of its success in other scientific contexts (e.g., sociology

or public health; Anderson et al., 2017; Wilson & Abbott, 2018).

FLYNN EFFECT IN ROMANIA 6

Method

Measures

The Multidimensional Aptitude Battery (MAB). We measured general cognitive

ability using the second edition of the Multidimensional Aptitude Battery (MAB, Jackson, 1998),

in its culturally adapted Romanian version (Iliescu et al., 2009). The MAB was developed with

the purpose of efficiently and objectively measuring general cognitive ability, as a paper-and-

pencil version of the Wechsler Adult Intelligence Scale, Revised (Jackson, 1998). The battery

consists of ten subtests of which five refer to components of verbal intelligence (summing into a

Verbal IQ score, VIQ) and five measure cognitive performance (summing into a Performance IQ

score, PIQ); these two sub-scores pool into a general measure of IQ (Full Scale IQ, FSIQ). The

verbal subscales are as follows: Vocabulary, indicating the number of stored verbal

concepts/words; Similarities, a measure of the capacity to discriminate and compare object

properties; Information, a proxy for the development of acquired knowledge, heavily dependent

on long-term memory; Arithmetic, requiring numerical reasoning and Comprehension, assessing

abilities that pertain to social situations (e.g., correct evaluation of norms, desired codes of

conduct). For the performance factor, the MAB-II uses the following subtests: Spatial, object

visualization and rotation, Picture Completion, identification of missing elements within images,

Picture Arrangement - a task linked to perceptual and adaptive skills, Object Assembly, another

visualization task, requiring perceptual analytical skills and lastly, Digit Symbol, assessing the

ability to learn and appropriately using new coding systems.

Generations. Before proceeding with examining potential generational effects, we first

have to define, or identify a generational taxonomy. While the structure of Howe and Strauss

(2000) is perhaps the better known taxonomy, it is hardly applicable to Romanian society for
FLYNN EFFECT IN ROMANIA 7

reasons of historical nature. Therefore, we identified and defined our own set of generations,

which we feel are more representative of Romanian society while, where possible, keeping close

to internationally defined generations. For example, we kept the definition of the Silent

Generation (starting with 1928 until 1945), as well as that of the Boomer Generation (starting

with the end of WWII). However, beginning with the year 1967 we identified the above-

mentioned “Decree” generation, which we define roughly up until 1977, where the fertility quota

reached rates prior to the ban and also roughly coincided with an approximate decade per

generation split. The 1977-1989 period was historically the last generation to be born under the

communist regime, in stark contrast to the 1989-1995 period, considered to be the first

generation born in truly democratic conditions. The last split in our data includes the period

1995-2008, a cohort that is much closer in education and attitudes to western generations. Our

analysis therefore assumes the existence of 7 generations: the Post-War Cohort (1928-1945), the

Boomer Generations (“early” 1946-1954, and “late” 1955-1967), the Decree Generation (1968-

1977), the Late-Communist Generation (1978-1989), the Revolution Generation (1990-1995),

and the Democratic Generation (1995-2008).

Sample and Procedure

The sample consists of 12034 participants, tested with the MAB between 2003 and 2018.

A number of 5990 (49.8%) participants were male, and ages ranged between 10 and 74 years (M

= 27.54, SD = 15.69). A number of 3200 of the 3435 participants collected in 2003 comprise the

Romanian standardization sample, a nationally representative sample aged 12-74 years with a

controlled distribution on geographic location, rural/urban location, gender, age and socio-

economic status (for details see Iliescu et al., 2009). All the other data were collected by

psychologists using the test in regular assessment work with their clients and shared with the
FLYNN EFFECT IN ROMANIA 8

Romanian test publisher for research purposes. The data was collected by psychologists in face-

to-face, paper-and-pencil administration. Most of the data pooled in our study was collected in

the case of adolescents in the context of assessment for vocational counseling and in the case of

adults in the context of personnel selection; we excluded all other data, e.g., data that was

collected in the context of clinical work. All the raw data was scored and transformed to IQ

scores based on the Romanian norms of the test (Iliescu et al., 2009).

Statistical Analysis

The variables that are needed for observation in the behavioral sciences are most of the

time simply unavailable as true panel data (i.e., repeated measures from the same individuals).

This has been especially true in FE studies. While there are some notable exceptions (such as

studies based on the National Longitudinal Survey of Youth, e.g., Rodgers & Wänström, 2007;

on the Survey of Health, Aging and Retirement in Europe, e.g., Weber et al., 2017, and others), a

large number of studies on the Flynn-Effect simply compare means obtained from non-

representative samples with often very low sample size (Bakhiet et al., 2014; Dutton & Lynn,

2015; Grigoriev & Lynn, 2014; Laciga & Cígler, 2017). Even when sample size is larger or

samples are representative, analytic sophistication is rather low and efforts to disentangle various

kinds of effects on the observed increase are rarely made (e.g., Dutton & Lynn, 2013) - again

with notable exceptions (e.g., Shiu et al., 2013).

One way in which this downside may be countered is through the use of pseudo-panel

methods. Pseudo-panels are pooled data from repeated cross-sectional measurements of the same

variable set over a period of time (see Deaton, 1985). However, unlike true panel data, the

tracked individuals are not the same across the repeated samples; instead pseudo-cohorts are

constructed, that represent the average life-experiences of individuals that share roughly the
FLYNN EFFECT IN ROMANIA 9

same birth period within the pooled data (Guillerm, 2017). For example, in our data, the 1977

birth cohort is represented by participants that are 30 years old in the data collected in 2007 and

by participants that are 40 years old in the data collected in 2017. There are several advantages to

pseudo-panels, such as the lack of non-random attrition. Even more importantly, we can consider

that the persons that build these cohorts share - aside from their age - influences from societal

factors, such as the common educational period in which they went to school, or factors that

pertain to the economical situations they were exposed to (e.g., same labor market conditions).

Therefore, the pseudo-cohort method is able to follow a certain generation as it moves through

time-bound experiences.

One method that is very well suited for the analysis of pseudo-panel data is the Age-

Period-Cohort (APC) analysis. APCs can be used to deconstruct and untie effects that are

attributable to either age, cohort or the time of the survey (data collection). Within APC

methodology, Age Effects refer to variations that are accounted for by the individual advancing

in chronological age. They mostly include effects that relate to changes in physiology and

performance (e.g., variations in peak cognitive functioning; Hartshorne & Germine, 2015) but

also describe changes in social experiences or social status. For example, changes in education,

employment or political views are such variables that can be modelled as a function of biological

aging. Period Effects are considered to be “variations over time periods or calendar years that

influence all age groups simultaneously” (Yang & Land, 2013, p. 2). They are generally

exogenous, with typical examples being changes in the environment, varying social or economic

factors, changes in labor market conditions or even drastic changes in state organization. Cohort

Effects define changes within or across groups that share an initial common event. Birth cohorts

are perhaps their most obvious and most used application, but there are other forms of cohorts,
FLYNN EFFECT IN ROMANIA 10

such as those built based on shared exposure to some external factor (e.g., environment

contamination). Individuals within the same cohort experience external changes at the same age.

Reforms in education or technological changes will have different effects for different cohorts.

For example, the exposure of cohorts born in 1990s vs. cohorts born in the 1960s to the internet

will have different outcomes.

Classical Age-Period-Cohort (APC) methods are not without drawbacks. Most

noteworthy are the multicollinearity issues that arise from the linear dependency between age,

period and cohort (i.e., Cohort = Period - Age), known as the “APC identification problem”.

While prior solutions are considered inadequate, one recent development has found common

scholarly acceptance by offering a more compelling conceptual framework (Reither et al., 2015).

Hierarchical APC estimation with cross-classified random effects modeling (HAPC-

CCREM; Yang & Land, 2013) analysis is a mixed (fixed and random effects) models approach

to the identification issue that is specifically intended for repeated cross-sectional data. In

contrast to the classic APC approaches where "… age, time period, and birth cohort are

considered same-level factors affecting the outcome of interest" (Yang & Land, 2013, p. 18),

HAPC methods view the repeated cross-sectional data as hierarchically structured, i.e., where the

individuals are nested within the newly built pseudo-cohorts and period of measurement groups.

HAPC, and mixed models in general, are normally used to control for correlated errors due to the

nested structure of the data. In our case, the data will present common variation due to

individuals belonging to the same birth cohort or to the same time of measurement.

Furthermore, HAPC also separates total variance into an individual, level-1 component

and two higher-level variance components, representing variance attributable to period and

respectively cohort effects. Since age is considered individual in nature, HAPC models treat it as
FLYNN EFFECT IN ROMANIA 11

a fixed-effect, individual-level variable, separating it from the period and cohort levels which

then are treated as random effects. This allows the age intercept to vary randomly by cohort and

period, thus avoiding the identification problem (Yang & Land, 2013).

Our analysis included both the “orthodox” approach of a regression analysis, and the

more innovative approach of the HAPC-CCREM analysis. In the regression analysis, we tested

for the predictive power of the year of testing on IQ and estimated the yearly growth of IQ per

year from the beta value of the predictor. In the HAPC-CCREM analysis, we disentangled the

overall age, period and generational variances and analyzed them separately.

Results

Table 1 depicts the summary of IQ scores (global, performance and verbal) based on the

year in which the data was collected. A visual inspection of IQ scores shows that all the yearly

samples have means that are position reasonably close around the IQ mean of the Romanian

population; also, it suggests a slight progression of mean scores, with higher mean scores in the

latter half of the sampled timeframe.

FLYNN EFFECT IN ROMANIA 12

Table 1

Sample Description by Year of Testing and Average IQ scores on the MAB-II

Year M Age SD Age Men Women IQ PIQ VIQ

2003 33.52 20.94 1707 1728 101.03 100.43 101.14
2004 25.02 15.12 345 348 100.56 99.70 100.92
2005 26.76 16.08 128 111 102.77 103.10 102.02
2006 23.84 13.77 157 136 100.63 100.11 100.66
2007 24.36 13.00 102 125 99.40 97.71 100.61
2008 26.41 14.68 476 428 103.08 102.57 103.10
2009 26.00 12.94 316 499 105.94 104.30 107.05
2010 24.28 10.65 252 256 104.13 103.08 104.67
2011 20.94 7.21 455 361 103.09 102.19 103.45
2012 24.44 11.03 641 497 103.63 102.80 103.97
2013 25.66 11.87 443 383 104.42 103.95 104.35
2014 26.81 6.81 174 201 106.94 106.10 107.25
2015 27.52 12.35 203 312 106.60 105.78 106.91
2016 27.50 12.38 229 262 105.65 104.75 106.02
2017 25.99 12.77 158 193 106.40 105.58 106.74
2018 23.64 8.88 204 204 102.55 102.72 101.91
Note. N = 12034; IQ = MAB-II IQ score. PIQ = Performance component of the MAB-II; VIQ =

Verbal component of the MAB-II.

Bivariate analysis

Various assumptions must be met before regression techniques can be applied (Cohen et

al., 2013). As a first step, we ran a residual analysis, which showed that the histogram of

standardized residuals displayed data that contained normally distributed errors. The normal P-P

plot of standardized residuals also displayed points quite close on the line, even if not absolutely

on it. Furthermore, the scatterplot of standardized residuals showed that the data met the

assumptions of homogeneity of variance and linearity.

FLYNN EFFECT IN ROMANIA 13

Having met the needed assumptions, we conducted a multiple regression to check if the

year of testing predicted IQ scores measured with the MAB-II. Table 2 shows the results for this

hierarchical multiple regression. In the first step, we introduced the control variables that were

followed in the second step by the predictor. We controlled for age as well as for gender. This

control block did not have predictive power on IQ scores: R2 = .20, 95%CI [0.00, 0.46], F(2,13)

= 1.65, p = .230. We then introduced the year of testing in the second step of the regression. The

associated change in R2 was significant, and the result indicated that the year of testing explained

a significant amount of the variance in MAB-II measured IQ scores with R2 = .61, 95%CI [0.07,

0.73], F(3,12) = 6.12, p = .009. Looking at the individual beta value of the predictor (b = 0.34,

95% CI [0.13, 0.56]), we can assert that the yearly estimate for IQ increase in our sample is

about a third of a full point, translating into an increase of around three IQ points per decade.

Such a development falls within the range of observed IQ increases in other samples around the

world.
FLYNN EFFECT IN ROMANIA 14

Table 2

Regression Results Using IQ as the Criterion

b beta sr2
2
Predictor b 95% CI beta 95% CI sr 95% CI r Fit Difference
[LL, UL] [LL, UL] [LL, UL]
Step 1
(Intercept) 73.44** [37.40, 109.48]
Age -0.03 [-0.52, 0.47] -0.03 [-0.59, 0.53] .00 [-.03, .03] .11
Gender 20.43 [-4.61, 45.47] 0.46 [-0.10, 1.02] .19 [-.15, .54] .45
R2 = .202
95% CI[.00,.46]
Step 2
(Intercept) -607.05** [-1031.96, -182.14]
Age 0.18 [-0.21, 0.57] 0.20 [-0.24, 0.64] .03 [-.08, .15] .11
Gender 9.57 [-10.13, 29.27] 0.21 [-0.23, 0.66] .04 [-.08, .16] .45
Year of Testing 0.34** [0.13, 0.56] 0.69 [0.26, 1.13] .40 [.05, .75] .70**
R2 = .605** ΔR2 = .403**
95% CI[.07,.73] 95% CI[.05, .75]

Note. A significant b-weight indicates the beta-weight and semi-partial correlation are also significant. b represents unstandardized
regression weights. beta indicates the standardized regression weights. sr2 represents the semi-partial correlation squared. r represents
the zero-order correlation. LL and UL indicate the lower and upper limits of a confidence interval, respectively.
* indicates p < .05. ** indicates p < .01.
FLYNN EFFECT IN ROMANIA 15

We also applied the same analysis to the principal MAB-II components, i.e., the verbal

and performance factors. The obtained results are reported in the Appendix and are similar to

those described above for the full-scale IQ score: for the verbal component, controlling for age

and gender, we observed a beta value of b = 0.37, 95%CI [0.14, 0.59] and for the performance

component a beta value of b = 0.39, 95%CI [0.19, 0.58], both significant.

Hierarchical Age-Period-Cohort (HAPC-CCREM) model

Moving beyond the bivariate analysis of the data, the next step explored and separated

the overall age (A), period (P) and generational/cohort (C) variance within the data. In order to

achieve this, we began by organizing the repeated cross-sectional data. Table 3 shows the two-

way cross-classified structure of the resulting data set, with the number of observations in each

cohort-by-period cell. However, before moving on to the full HAPC analysis, we wanted to rule

out that the data could be explained by any of the single factors (age, period or cohort). The

results of these analyses are presented in tables 4 to 6. They indicate that each separate factor

seems to be operative in our sample. For example, age has a negative, very small, non-linear yet

significant effect on verbal, performance and overall IQ scores. Furthermore, period (R2 = .02,

95%CI [0.01, 0.03], F(15,12018) = 17.48, p < .001) and cohort effects (R2 = .01, 95%CI [0.01,

0.01], F(6,12027) = 18.42, p < .001) significantly explained a proportion of the variance. Model

fit comparisons using maximum likelihood ratios between A, P and C models individually, as

well as comparing increasing A, AP, AC, and APC models showed that the latter model is the

best representation of the sampled data.

FLYNN EFFECT IN ROMANIA 16

Table 3: Two-Way Cross-Classified Data Structure in the Pooled Dataset. Number of Observations in Each Cohort-by-Period Cell

Cohort 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 Total
1928 - 1945 720 42 15 12 8 40 15 0 1 7 2 0 0 0 0 0 862

1945 - 1954 235 27 13 5 3 22 11 4 2 21 17 0 15 9 4 0 388

1954 - 1967 422 61 27 27 15 86 90 36 11 46 54 3 20 27 19 1 945

1967 - 1977 271 64 24 27 26 114 109 63 47 103 71 33 56 47 32 22 1109

1977 - 1989 1252 334 93 112 94 287 212 173 150 278 196 183 150 146 66 87 3813

1989 - 1995 535 165 67 107 74 283 311 136 511 466 281 142 124 95 63 76 3436

1995 - 2008 0 0 0 3 7 72 67 96 94 217 205 14 150 167 167 222 1481

Total 3435 693 239 293 227 904 815 508 816 1138 826 375 515 491 351 408 12034
FLYNN EFFECT IN ROMANIA 17

Table 4: Bivariate Regression Model, Age Effect

Dependent variable:
IQ Performance Scores Verbal Scores
(1) (2) (3)
Age 0.160*** 0.177*** 0.142***
(0.039) (0.044) (0.043)
Age2 -0.003*** -0.003*** -0.003***
(0.0005) (0.001) (0.001)
Constant 101.386*** 100.308*** 101.961***
(0.598) (0.674) (0.668)
Observations 12,034 12,034 12,034
2
R 0.005 0.004 0.004
2
Adjusted R 0.005 0.003 0.004
Residual Std. Error (df = 12031) 14.188 15.991 15.848
F Statistic (df = 2; 12031) 30.510*** 22.017*** 27.081***
Note: ∗p<0.1; ∗∗p<0.05; ∗∗∗p<0.01
FLYNN EFFECT IN ROMANIA 18

Table 5: Bivariate Regression Model, Period Effects

Dependent variable:
IQ Performance Scores Verbal Scores
(1) (2) (3)
Period
2004 -0.475 -0.728 -0.216
(0.586) (0.662) (0.655)
2005 1.739* 2.669** 0.885
(0.942) (1.064) (1.053)
2006 -0.399 -0.318 -0.480
(0.857) (0.968) (0.958)
2007 -1.634* -2.713** -0.523
(0.965) (1.090) (1.078)
2008 2.051*** 2.138*** 1.964***
(0.526) (0.595) (0.588)
2009 4.908*** 3.878*** 5.916***
(0.549) (0.620) (0.613)
2010 3.101*** 2.650*** 3.532***
(0.669) (0.756) (0.748)
2011 2.055*** 1.759*** 2.319***
(0.548) (0.619) (0.613)
2012 2.596*** 2.369*** 2.832***
(0.482) (0.544) (0.538)
2013 3.391*** 3.526*** 3.217***
(0.546) (0.616) (0.610)
2014 5.911*** 5.672*** 6.118***
(0.766) (0.865) (0.856)
2015 5.573*** 5.354*** 5.775***
(0.665) (0.752) (0.744)
2016 4.621*** 4.325*** 4.881***
(0.679) (0.767) (0.759)
2017 5.374*** 5.154*** 5.605***
(0.789) (0.891) (0.882)
2018 1.516** 2.289*** 0.771
(0.737) (0.833) (0.824)
Constant 101.031*** 100.427*** 101.136***
(0.240) (0.271) (0.268)
Observations 12,034 12,034 12,034
2
R 0.021 0.015 0.019
Adjusted R2 0.020 0.014 0.018
Residual Std. Error (df = 12018) 14.079 15.904 15.736
F Statistic (df = 15; 12018) 17.480*** 12.576*** 15.922***
Note: ∗p<0.1; ∗∗p<0.05; ∗∗∗p<0.01
FLYNN EFFECT IN ROMANIA 19

Table 6: Bivariate Regression Model, Generation Effects

Dependent variable:
IQ Performance Scores Verbal Scores
(1) (2) (3)
Generations
1945, 1954 -0.475 -0.142 -0.835
(0.866) (0.977) (0.967)
1954, 1967 0.604 0.695 0.508
(0.667) (0.753) (0.745)
1967, 1977 3.454*** 2.870*** 4.046***
(0.643) (0.726) (0.718)
1977, 1989 3.699*** 3.411*** 3.996***
(0.534) (0.603) (0.596)
1989, 1995 2.929*** 2.839*** 3.004***
(0.539) (0.609) (0.602)
1995, 2008 4.261*** 3.861*** 4.652***
(0.607) (0.684) (0.677)
Constant 100.168*** 99.629*** 100.206***
(0.482) (0.544) (0.539)
Observations 12,034 12,034 12,034
2
R 0.009 0.006 0.009
2
Adjusted R 0.009 0.005 0.009
Residual Std. Error (df = 12027) 14.161 15.978 15.812
F Statistic (df = 6; 12027) 18.425*** 11.250*** 18.862***
Note: ∗p<0.1; ∗∗p<0.05; ∗∗∗p<0.01
FLYNN EFFECT IN ROMANIA 20

Table 7: Hierarchical Age-Period-Cohort model (HAPC-CCREM) of the IQ Data

Variable IQ Verbal Performance

Intercept 103.950 0.849 102.451 0.938 105.025 0.835

Age
Age -0.113 0.390 -0.531 0.445 0.205 0.387
Age2 -0.088 0.209 0.131 0.235 -0.300 0.218
Gender -0.510 0.258 0.539 0.288 -1.555 0.291

Period
2003 -2.214 0.411 -2.431 0.463 -2.022 0.401
2004 -2.942 0.611 -2.913 0.681 -2.919 0.647
2005 -0.632 0.889 -1.475 0.984 0.265 0.959
2006 -2.617 0.827 -2.786 0.916 -2.322 0.891
2007 -3.626 0.908 -2.834 1.003 -4.125 0.979
2008 -0.423 0.559 -0.644 0.624 -0.211 0.585
2009 2.417 0.579 3.124 0.646 1.628 0.609
2010 0.431 0.678 0.663 0.754 0.164 0.724
2011 -0.429 0.589 -0.163 0.657 -0.702 0.620
2012 0.052 0.526 0.262 0.588 -0.140 0.546
2013 0.859 0.578 0.645 0.644 1.022 0.607
2014 2.786 0.759 2.906 0.843 2.541 0.816
2015 2.836 0.675 2.807 0.752 2.766 0.721
2016 1.904 0.688 1.996 0.766 1.728 0.735
2017 2.612 0.781 2.611 0.866 2.508 0.839
2018 -1.015 0.747 -1.768 0.829 -0.183 0.800

Cohort
1928 - 1945 -0.107 0.540 -0.265 0.613 0.139 0.533
1945 - 1954 -1.119 0.639 -1.319 0.728 -0.730 0.620
1954 - 1967 -1.066 0.511 -0.993 0.578 -0.997 0.504
1967 - 1977 0.664 0.489 1.284 0.552 0.050 0.483
1977 - 1989 1.302 0.395 1.398 0.444 1.086 0.378
1989 - 1995 0.019 0.403 -0.346 0.453 0.228 0.389
1995 - 2008 0.307 0.479 0.242 0.539 0.223 0.472

Variance Variance se Variance se Variance se

Components
Period 4.944 2.224 5.393 2.322 4.6590 2.1585
Cohort 1.097 1.047 1.525 1.235 0.7682 0.8765
Residual 197.384 14.049 246.466 15.699 251.819 15.868
FLYNN EFFECT IN ROMANIA 21

The full model is reported in Table 7, reporting the parameter estimates and model fit

statistics for the HAPC-CCREM on the 16-repeated cross-section datasets. These estimates

are obtained through restricted maximum-likelihood methods (REML) delivered by the lme4

package (Bates et al., 2015) in R (R Core Team, 2014). For better ease of usage and better

understanding of the results, we graphically plotted the estimated cohort and period effect

coefficients separately, as showed by Frenk, Yang and Lang (2013). For example, Figure 1

shows the period IQ score effects averaged over all cohorts for each time period, whereas

Figure 2 plots the estimated cohort IQ score effects averaged over all time periods for each

cohort. Figure 3 plots the estimated generation IQ score effects. In all figures, the overall

estimated intercept is drawn at value 0 (i.e., 103.95 IQ Points in our data) making any

significant deviations from this value (by either cohort or period estimates) more readily

observable. Confidence intervals (95%) are also reported through the dotted lines around

estimates. A more detailed composition of the mean IQ scores is also reported: the verbal and

performance components of the MAB-II are reported in Figures 2 and 3 using the same

principles as described above.

FLYNN EFFECT IN ROMANIA 22

Figure 1. Age effects on the IQ scores measured through the MAB II, HAPC-CCREM model.
FLYNN EFFECT IN ROMANIA 23

Figure 2. Period effects on the MAB II Components, HAPC-CCREM model.

FLYNN EFFECT IN ROMANIA 24

Figure 3. Generation effects on the MAB II Components, HAPC-CCREM model.

Results indicated that in our sample, we can find evidence for sizeable differences

between generations. For example, when controlled for age and period effects, post-war

generations up until (and including) the 1967-generation (the “Decree” generation) lag behind

all other following generations. Following the normalization of the fertility rate (ca. 1977) the

mean IQ scores begin to increase mainly driven by an increase in the verbal component of the

test, slowly followed by the performance component. The peak is reached with the

“Revolution” generation, where both MAB-II components achieve their maximum, followed

by a steady decrease, with the 1989-1994 and 1995-2008 cohorts reaching values close to
FLYNN EFFECT IN ROMANIA 25

global intercept. Interestingly, IQ components also change their primacy over birth cohorts.

The verbal component takes over as the main driver of IQ increase for the 1977 and 1989

generations and then relinquishes it for the subsequent period (see Figure 3).

Moving on to period effects, Figure 2 indicates that the steady increase in IQ scores is

partly attributable to this variance; we find it however impossible to point with clarity to any

variable responsible for this effect. As previously mentioned, period effects are generally

exogenous: they could subsume social or economic factors, environmental changes and

others. With small variations, the estimates of the HAPC-CCREM model for the period

effects show that the data collected after 2007 surpass the overall intercept estimate. One last

glimpse over the figures shows (Figure 1) that age has a decreasing and nonlinear effect.

Concerning the estimated age effects, their pattern is not rich in variability; while it suggests a

small decline in IQ score towards older participants, the decline is rather small (about one IQ

point for an age range of several decades).

Discussion

The data stemming from 12034 Romanian participants with ages 10 to 74 years tested

with the same cognitive ability measure between 2003 and 2018 points to the fact that the

Romanian population has undergone a positive Flynn effect of about 3 points per decade. The

more sophisticated statistical approach of Hierarchical APC Estimation with Cross-Classified

Random Effects Modeling (HAPC-CCREM) disentangles the effects of chronological age in

participants, year of testing and cohort membership, and points to the fact that all these three

components contribute to some extent to the variance on data and to the detected Flynn effect.

However, we stress that the variation in IQ scores is mainly individual, with only a low

percent of the variance attributable to both period and cohort effects, with no almost no age

effect.

We have sizeable differences between generations, with a peak IQ in the 1990-1995

cohort, followed by a slight decrease afterwards. If this suggested decrease will continue,
FLYNN EFFECT IN ROMANIA 26

Romania may well report in future studies a negative Flynn effect for the current and

following decades. Cohort-effects can in general easily be explained by societal changes and

upheavals, and the Romanian data clearly fits this rule. The peak in IQ scores coincides with

that time period when the government dedicated consistent spending to education (the “Late-

Communist” generation, i.e., 1978-1989), with a steady decline afterwards. This result offers

some support to the hypothesis championed by Rönnlund & Nilsson (2008) that education

might be a part of the antecedents for the FE. If this is true, then the expectations for future

reports of a negative Flynn effect in Romania are even more consistent, as the Romanian

educational system has struggled through ill-devised reforms and low academic performance

for the past at least 20 years now. This is a matter of national concern and is also proven by

the results of international comparative studies in education. For example, the results for the

2015 edition of the Programme for International Student Assessment (PISA) of the

Organization for Economic Cooperation and Development (OECD) place Romania on place

48 of 70 participant countries in the Science assessment (the main domain for PISA 2015),

with 435 points, well below the OECD average of 493 points. The positions for Math and

Literacy are quite similar and place Romania on one of the very last places in Europe (OECD,

2016).

Yet another variable that could play a role in explaining the negative difference

between the two post-1989 cohorts (those labeled by us the “Revolution Generation” and the

“Democratic Generation”) is the massive migration levels experienced by Romania in the past

decades. According to new Eurostat figures, almost every fifth working-age Romanian lives

in the European Union (Statistical Office of the European Communities, 2018). This trend

started immediately after the fall of communism and was only amplified by the EU admission

in 2007. As a consequence, the Romanian “homeland” population decreased from 23.2M in

1989 to 19.8M in 2015, at a rate of 7.3% per year (observed between 2000 and 2015) –

among the highest in the world (Iacob, 2018). Incidentally, this phenomenon was driven
FLYNN EFFECT IN ROMANIA 27

primarily by high-skilled individuals; as 2015 data indicates approximately 480.000 of such

workers have left Romania in each year between the years 1997 and 2013 (Iacob, 2018). For

example, according to a World Bank report, more than a quarter of the total number of

Romanian physicians had already left the country by the year 2013 (The World Bank, 2018).

Therefore, we argue that these highly skilled individuals systematically exiting the sample

population could help in explaining our results. Such an interpretation would also be

consistent with recent findings showing a link between brain drain caused by the ongoing

civil war in Syria and a stop of the Flynn Effect in Damascus (Dutton et al., 2018).

Period (time of testing) effects are also present, but are not clearly attributable to a

specific variable. They have an erratic pattern, with humps and dips, but show by and large an

increase by time of testing, that could be explained by the population acquiring test-taking

skills, or having greater test familiarity, having access to higher quality resources (diet,

nutrients), or access to information (ubiquity of personal computers and widespread internet

usage) – all of them having been consistently linked to increases in test scores in previous

research.

Age effects suggests a small decline in IQ scores towards older participants. This

decline is trivial (about one IQ point for an age range of several decades) and is most likely an

effect of cognitive decline with age. The perceived decline is also non-linear, suggesting

acceleration towards larger age groups; however, the rather limited samples sizes for the older

age groups makes a dedicated analysis of this accelerated decline impossible. We nevertheless

assert that age decline, while minuscule, is consistent with the literature on age differences in

intelligence and cognitive decline (e.g., Park, O'Connell, & Thomson, 2003).

We would like to emphasize the utility of the more specialized Hierarchical Age-

Periods-Cohort (HAPC) models in the study of the FE. Some of the studies reporting on the

FE suffer from serious methodological weaknesses, oftentimes reporting nothing else than

simple t tests of the average scores, in unconvincing samples. Different-time samples used in
FLYNN EFFECT IN ROMANIA 28

such studies are sometimes claimed to be representative (although proof is rarely shown), are

usually not compared in terms of equivalence to each other and are many a time small in

volume. Simple comparison of averages (t tests), instead of more sophisticated latent mean

comparisons, latent growth models or time series are used. All this is a function of the

difficulty to provide convincing samples containing true panel data (i.e., repeated measures

from the same individuals, over time). HAPC permits the analysis of pseudo-panels, making it

possible for researchers to pool data together from various samples collected over time with

one or several cognitive measures and makes it thereby possible to have reports of the FE

from countries that could not otherwise contribute to this scientific discussion.

Limitations

As with all studies, there are both limitations and strengths to our undertaking. An

important limitation is that our sample, while comparatively large, does not conform to true-

panel specifications. While desirable, true panel data is especially difficult to obtain. We do

believe that the usage of pseudo-panel data coupled with a suitable methodological approach

is more than adequate in our situation. Second, with the exception of the 2013 data, that was

based on a dedicated data collection process with a controlled distribution on a number of key

variables, all the other data was obtained from psychologists working in various populations

and contexts, and we had no control over the sampling or test administration.

Conclusion

The present study contributes to the literature on the Flynn effect by providing results

showing a positive Flynn effect from an Eastern European culture from where data were

previously not reported. At the same time, the study proposes a method of analysis (HAPC-

CCREM) that disentangles the effects of chronological age (Age), year of testing (Period) and

generation (Cohort), thereby making the analysis of pooled samples possible, potentially

increasing the repertoire of methods employed in the analysis of the FE.

FLYNN EFFECT IN ROMANIA 29

References

Anderson, A. L., Lytle, R., & Schwadel, P. (2017). Age, period, and cohort effects on death

penalty attitudes in the united states, 1974-2014: HAPC effects on death penalty

attitudes. Criminology, 55(4), 833–868. https://doi.org/10.1111/1745-9125.12160

Bakhiet, S. eldin F. A., Barakat, S. M. R., & Lynn, R. (2014). A Flynn effect among deaf

boys in Saudi Arabia. Intelligence, 44, 75–77.

https://doi.org/10.1016/j.intell.2014.03.003

Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015). Fitting linear mixed-effects models

using lme4. Journal of Statistical Software, 67(1).

https://doi.org/10.18637/jss.v067.i01

Bjerkedal, T., Kristensen, P., Skjeret, G. A., & Brevik, J. I. (2007). Intelligence test scores

and birth order among young Norwegian men (conscripts) analyzed within and

between families. Intelligence, 35(5), 503–514.

https://doi.org/10.1016/j.intell.2007.01.004

Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2013). Applied multiple regression and

correlation analysis for the behavioral sciences (3rd ed.). Routledge.

https://doi.org/10.4324/9780203774441

Cotton, S. M., Kiely, P. M., Crewther, D. P., Thomson, B., Laycock, R., & Crewther, S. G.

(2005). A normative and reliability study for the Raven’s Coloured Progressive

Matrices for primary school aged children from Victoria, Australia. Personality and

Individual Differences, 39(3), 647–659. https://doi.org/10.1016/j.paid.2005.02.015

Deaton, A. (1985). Panel data from time series of cross-sections. Journal of Econometrics,

30(1–2), 109–126. https://doi.org/10.1016/0304-4076(85)90134-4

Dragolea, A. (2007). Dimensiunea de gen a pietei muncii [The gender dimension of the labor

market]. In Parteneri Egali. Competitori Egali [Equal Partners. Equal Competitors]

(pp. 27–83). Maiko.

FLYNN EFFECT IN ROMANIA 30

Dutton, E., Essa, Y. A. S., Bakhiet, S. F., Ali, H. A. A., Alqafari, S. M., Alfaleh, A. S. H., &

Becker, D. (2018). Brain drain in Syria’s ancient capital: No Flynn Effect in

Damascus, 2004–2013/14. Personality and Individual Differences, 125, 10–13.

https://doi.org/10.1016/j.paid.2017.12.025

Dutton, E., & Lynn, R. (2013). A negative Flynn effect in Finland, 1997–2009. Intelligence,

41(6), 817–820. https://doi.org/10.1016/j.intell.2013.05.008

Dutton, E., & Lynn, R. (2015). A negative Flynn Effect in France, 1999 to 2008–9.

Intelligence, 51, 67–70. https://doi.org/10.1016/j.intell.2015.05.005

Frenk, S. M., Yang, Y. C., & Land, K. C. (2013). Assessing the significance of cohort and

period effects in hierarchical age-period-cohort models: Applications to verbal test

scores and voter turnout in u.s. presidential elections. Social Forces, 92(1), 221–248.

https://doi.org/10.1093/sf/sot066

Grigoriev, A., & Lynn, R. (2014). A study of the intelligence of Kazakhs, Russians and

Uzbeks in Kazakhstan. Intelligence, 46, 40–46.

https://doi.org/10.1016/j.intell.2014.05.004

Guillerm, M. (2017). Les méthodes de pseudo-panel et un exemple d’application aux données

du patrimoine. Economie et Statistique / Economics and Statistics, 491–492, 119–140.

https://doi.org/10.24187/ecostat.2017.491d.1908

Hartshorne, J. K., & Germine, L. T. (2015). When does cognitive functioning peak? The

asynchronous rise and fall of different cognitive abilities across the life span.

Psychological Science, 26(4), 433–443. https://doi.org/10.1177/0956797614567339

Howe, N., & Strauss, W. (2000). Millennials rising: The next great generation. Vintage

Books.

Iacob, R. (2018). Brain drain phenomenon in Romania: What comes in line after corruption?

Romanian Journal of Communication and Public Relations, 20(2), 53.

https://doi.org/10.21018/rjcpr.2018.2.259
FLYNN EFFECT IN ROMANIA 31

Iliescu, D., Glinta, F., & Ispas, D. (2009). Adaptarea culturala a MAB-II (Multidimensional

Aptitude Battery) în Romania [Cultural adaptation of the MAB-II in Romania].

Psihologia Resurselor Umane, 7(1), 75–87.

Jackson, D. N. (1998). Multidimensional Aptitude Battery–II: Manual. Sigma Assessment

Systems.

Kowske, B. J., Rasch, R., & Wiley, J. (2010). Millennials’ (lack of) attitude problem: An

empirical examination of generational effects on work attitudes. Journal of Business

and Psychology, 25(2), 265–279. https://doi.org/10.1007/s10869-010-9171-8

Laciga, J., & Cígler, H. (2017). The Flynn effect in the Czech Republic. Intelligence, 61, 7–

10. https://doi.org/10.1016/j.intell.2016.11.005

Must, O., & Must, A. (2013). Changes in test-taking patterns over time. Intelligence, 41(6),

780–790. https://doi.org/10.1016/j.intell.2013.04.005

OECD. (2016). PISA 2015 results (Volume I): Excellence and equity in education.

Pietschnig, J., Voracek, M., & Formann, A. K. (2010). Pervasiveness of the IQ rise: A cross-

temporal meta-analysis. PLoS ONE, 5(12), e14406.

https://doi.org/10.1371/journal.pone.0014406

R Core Team. (2014). R: A Language and Environment for Statistical Computing. R

Foundation for Statistical Computing. http://www.R-project.org/

Reither, E. N., Land, K. C., Jeon, S. Y., Powers, D. A., Masters, R. K., Zheng, H., Hardy, M.

A., Keyes, K. M., Fu, Q., Hanson, H. A., Smith, K. R., Utz, R. L., & Claire Yang, Y.

(2015). Clarifying hierarchical age–period–cohort models: A rejoinder to Bell and

Jones. Social Science & Medicine, 145, 125–128.

https://doi.org/10.1016/j.socscimed.2015.07.013

Rodgers, J. L. (1998). A critique of the Flynn Effect: Massive IQ gains, methodological

artifacts, or both? Intelligence, 26(4), 337–356. https://doi.org/10.1016/S0160-

2896(99)00004-5
FLYNN EFFECT IN ROMANIA 32

Rodgers, J. L., & Wänström, L. (2007). Identification of a Flynn Effect in the NLSY: Moving

from the center to the boundaries. Intelligence, 35(2), 187–196.

https://doi.org/10.1016/j.intell.2006.06.002

Rönnlund, M., & Nilsson, L.-G. (2008). The magnitude, generality, and determinants of

Flynn effects on forms of declarative memory and visuospatial ability: Time-

sequential analyses of data from a Swedish cohort study. Intelligence, 36(3), 192–209.

https://doi.org/10.1016/j.intell.2007.05.002

Shiu, W., Beaujean, A. A., Must, O., te Nijenhuis, J., & Must, A. (2013). An item-level

examination of the Flynn effect on the National Intelligence Test in Estonia.

Intelligence, 41(6), 770–779. https://doi.org/10.1016/j.intell.2013.05.007

Smola, K. W., & Sutton, C. D. (2002). Generational differences: Revisiting generational work

values for the new millennium. Journal of Organizational Behavior, 23(4), 363–382.

https://doi.org/10.1002/job.147

Statistical Office of the European Communities. (2018). EU citizens in other EU Member

States. EUROSTAT. https://ec.europa.eu/eurostat/documents/2995521/8926076/3-

28052018-AP-EN.pdf/48c473e8-c2c1-4942-b2a4-5761edacda37

Sundet, J. M., Barlaug, D. G., & Torjussen, T. M. (2004). The end of the Flynn effect?: A

study of secular trends in mean intelligence test scores of Norwegian conscripts during

half a century. Intelligence, 32(4), 349–362.

https://doi.org/10.1016/j.intell.2004.06.004

The World Bank. (2018). From uneven growth to inclusive development: Romania’s path to

shared prosperity.

http://documents.worldbank.org/curated/en/673121528867676119/pdf/127129-PUB-

PUBLIC-pub-date-6-6-18.pdf

Tudosoiu, S.-N. (2010). Educaţia în sistemul comunist din România (anii 1965-1989)

[Education in communist Romania (1965-1989)]. University of Bucharest.

 FLYNN EFFECT IN ROMANIA 33

Twenge, J. M., & Campbell, W. K. (2001). Age and birth cohort differences in self-esteem: A

cross-temporal meta-analysis. Personality and Social Psychology Review, 5(4), 321–

344. https://doi.org/10.1207/S15327957PSPR0504_3

Weber, D., Dekhtyar, S., & Herlitz, A. (2017). The Flynn effect in Europe – Effects of sex

and region. Intelligence, 60, 39–45. https://doi.org/10.1016/j.intell.2016.11.003

Weiss, L. G., Gregoire, J., & Zhu, J. (2016). Flaws in flynn effect research with the wechsler

scales. Journal of Psychoeducational Assessment, 34(5), 411–420.

https://doi.org/10.1177/0734282915621222

Williams, R. L. (2013). Overview of the Flynn effect. Intelligence, 41(6), 753–764.

https://doi.org/10.1016/j.intell.2013.04.010

Wilson, R., & Abbott, J. H. (2018). Age, period and cohort effects on body mass index in

New Zealand, 1997-2038. Australian and New Zealand Journal of Public Health,

42(4), 396–402. https://doi.org/10.1111/1753-6405.12804

Yang, Y., & Land, K. (2013). Age-Period-Cohort Analysis: New Models, Methods, and

Empirical Applications (Vol. 20131045). Chapman and Hall/CRC.

https://doi.org/10.1201/b13902

View publication stats