Korespondensi SosHum

BUKTI KORESPONDENSI
ARTIKEL JURNAL INTERNASIONAL BEREPUTASI
Judul artikel : Development and Initial Validation of Perceived Research

Environment Scale for Higher Education Academics
Jurnal : Journal of Psychoeducational Assessment, 2020, volume
38(2), 195 – 208
Penulis :
No. Perihal Tanggal
1. Bukti konfirmasi submit artikel dan artikel yang 26 Juni 2018

disbumit
2. Bukti konfirmasi review dan hasil review pertama 7 Okt 2018
3. Bukti konfirmasi submit revisi pertama, respon 3 Nov 2018
kepada reviewer, dan artikel yang diresubmit
4. Bukti konfirmasi review dan hasil review kedua 29 Des 2018
5. Bukti konfirmasi submit revisi kedua, respon kepada 2 Jan 2019
reviewer, dan artikel yang diresubmit
6. Bukti konfirmasi artikel accepted 16 Jan 2019
7. Bukti konfirmasi artikel published online 11 Feb 2019
1. Bukti Konfirmasi Submit Artikel
dan Artikel yang Disubmit
(26 Juni 2018)
8/1/2020 Gmail - Journal of Psychoeducational Assessment - Manuscript ID JPA-18-0139
Journal of Psychoeducational Assessment - Manuscript ID JPA-18-0139

Journal of Psychoeducational Assessment <onbehalfof@manuscriptcentral.com> Tue, Jun 26, 2018 at 12:23 PM
Reply-To: JPA@sagepub.com
To:
26-Jun-2018
Dear Dr. :
Your manuscript entitled "Development and Initial Validation of Perceived Research Environment Scale
for Higher Education Academics" has been successfully submitted online and is presently being given full
consideration for publication in Journal of Psychoeducational Assessment.
Your manuscript ID is JPA-18-0139.
Please mention the above manuscript ID in all future correspondence or when calling the office for questions. If there
are any changes in your street address or e-mail address, please log in to ScholarOne Manuscripts at
https://mc.manuscriptcentral.com/jopa and edit your user information as appropriate.
You can also view the status of your manuscript at any time by checking your Author Center after logging in to
https://mc.manuscriptcentral.com/jopa.
As part of our commitment to ensuring an ethical, transparent and fair peer review process SAGE is a supporting
member of ORCID, the Open Researcher and Contributor ID (https://orcid.org/). We encourage all authors and co-
authors to use ORCID iDs during the peer review process. If you already have an ORCID iD you can link this to your
account in ScholarOne just by logging in and editing your account information. If you do not already have an ORCID
iD you may login to your ScholarOne account to create your unique identifier and automatically add it to your profile.
Thank you for submitting your manuscript to Journal of Psychoeducational Assessment.
Sincerely,
Journal of Psychoeducational Assessment Editorial Office
https://mail.google.com/mail/u/0?ik=265fb369de&view=pt&search=all&permmsgid=msg-f%3A1604311437482278362&simpl=msg-f%3A16043114… 1/1
Journal of Psychoeducational Assessment
Development and Initial Validation of Perceived Research

Environment Scale
for Higher Education Academics
Journal: Journal of Psychoeducational Assessment

Fo
Manuscript ID Draft
Manuscript Type: Regular Article

r
perceived research environment, scale development, academics,

Keywords:
university, higher education
Pe
There is a growing interest in the perceived research environment for

higher education academics. As there is no existing, psychometrically
sound scale that directly measures perceived research environment for
er
higher education academics, we designed and validated the Perceived

Research Environment Scale for use with this population. In Phase 1, items
were developed based on a review of literature, six focus groups, and
Abstract: expert judgment. In Phase 2, the items were then administered to a
Re
sample of Indonesian academics (N = 306, M age = 42.29 years). Item

analysis and exploratory factor analysis were used to reduce the number of
items and determine the factor structure. In Phase 3, confirmatory factor
v
analyses were used on a hold-out sample (N = 292, M age = 43.39) to

confirm this structure. In Phase 4, we provided evidence for construct
iew
validity. The practical uses of this newly-developed scale are discussed.
http://mc.manuscriptcentral.com/jpa
Page 1 of 22 Journal of Psychoeducational Assessment
1
1
2
3 Development and Initial Validation of Perceived Research Environment Scale
4
5
6
7
8 Introduction
9
10 Education, research, and services are the three key functions characterizing the
11
12 academic profession in modern-day, higher education systems (Eam, 2015), although
13
14 academic research and publications have been increasingly emphasised at most universities
15
16 around the world, as involvement in research-related activities is recognised as an effective
17
18
means to upgrade a university’s profile (Nguyen, Klopper, & Smith, 2016). Previous studies
19
Fo
20
21 have demonstrated that engagement in research potentially improves teaching quality and
22
23 enhances knowledge and competence that contributes to high quality research supervision,
r
24
Pe
25 which is critical for developing graduate students as independent researchers (Lindsay,

26
27 Breen, & Jenkins, 2002).
28
er
29 Reflecting this, there has been a continuing trend for universities in developed
30
31
Re
countries to increase their focus on research, and this tendency has spread to developing
32
33
34
countries, where research is increasingly viewed as a high priority (Nguyen et al., 2016).
v
35
36 Consequently, research has become an important function for academics everywhere, as
iew
37
38 research productivity is now a primary consideration in several important organisational
39
40 decisions, such as hiring, maintenance of tenure, promotions, and salary increases for
41
42 academics (Chen, Gupta, & Hoshower, 2006). As academics are required to publish their
43
44 research results nationally and internationally in high quality, peer-refereed journals (Nguyen
45
46
et al., 2016), researchers have been interested in identifying the predictors of research
47
48
49 involvement and performance in academics (e.g., Whelan & Markless, 2013).
50
51 This research has shown that, among the factors that influence research productivity,
52
53 environmental factors are some of the most powerful ones (Bland & Ruffin, 1992), which has
54
55 lead researchers to identify the elements that characterise a good research environment (e.g.,
56
57
58
59
60 http://mc.manuscriptcentral.com/jpa
Journal of Psychoeducational Assessment Page 2 of 22
2
1 PERCEIVED RESEARCH ENVIRONMENT SCALE
2
3 Holden, Pager, Golenko, & Ware, 2012). Owen (1992) identified four important components
4
5
of a good research environment: research funding, research infrastructure, having active
6
7
8 researchers, and the availability of supportive research administrators. White, James, Burke,
9
10 and Allen (2012) demonstrated that a good research environment is one that provides
11
12 opportunities to access key resources and support from colleagues and the wider university.
13
14 More recently, Nguyen et al. (2016) conducted a qualitative study in Vietnam to examine the
15
16 affordances, barriers, and motivations that drive research engagement in academics. Financial
17
18 support for research activities was perceived as the main affordance, teaching load as the
19
Fo
20
21
main hindrance, and having a collaborative research environment and supportive research
22
23 policy settings and practices as motivations for academics to engage in research.
rP
24
25 The important role of the research environment is consistent with Bronfenbrenner’s
26
ee
27 (1979) ecological systems theory. The behaviour of academic peers can be considered a
28
29 “micro-environment”, which is embedded in larger environments, such as the university
rR
30
31 department or research centre, which, in turn, are influenced by university-level policies and
32
ev
33
resources. The micro-environment comprises the academic’s mentors’ and colleagues’
34
35
36 behaviours (i.e., role models and collaborators), other aspects of the department-level
iew
37
38 environment (e.g., requirements and recognition), and the university-level environment (e.g.,
39
40 incentives and policy guidelines; Mallinckrodt, 1997).
41
42 In the nursing area, Pranulis and Gortner (1985) identified several characteristics of
43
44 high productivity university departments: faculty competent in research skills, research
45
46
valued as desirable outcome goal, role responsibilities included time for faculty to engage in
47
48
49 research activities, compatibility between faculty research activities and organisational
50
51 mission and goals, as reflected in the support and rewards for research, support for, and
52
53 encouragement of, faculty’s efforts to seek extramural funding for research, administrative
54
55
56
57
58
59
3
2
3 support for research, and a psychosocial climate supportive of research and beginning
4
5
investigators.
6
7
8 In the area of psychology, Duffy et al. (2013) interviewed 17 of the most research-
9
10 productive counselling psychologists within the American Psychological Association
11
12 accredited counselling psychology program. Participants who were successful in their early
13
14 career had received mentorship/support while in graduate school, chose research topics that
15
16 were salient to them and they were passionate about, managed their time effectively, had
17
18 good collaboration with students and professionals, had structured strategies for writing,
19
Fo
20
21
worked in a supportive environment, and spent a considerable amount of time outside of
22
23 work with family or participating in hobbies. The work environment (i.e., participants’ day-
rP
24
25 to-day experiences at work) was identified as one of the main contributing factors. However,
26
ee
27 despite this evidence regarding the importance of the research environment, no measure has
28
29 been created specifically to assess the perceived research environment (PRE) of university
rR
30
31 academics.
32
ev
33
34
Previous Measures of Research-Related Environments
35
36 While there is no scale suitable to measure the research environment from the
iew
37
38 perspective of academics in a university setting, Young and Rice (1983) devised The
39
40 Research Environment Scale, a 24-item, six-point, Likert-type scale, specifically for nurses.
41
42 This scale measures aspects of the clinical research environment, such as educational
43
44 opportunities for nurses to learn about the research process. A sample item: “Nurses have
45
46
qualified mentors for conducting research”. Marsh and Brown (1992) removed six items that
47
48
49 were unrelated to the clinical research setting, and used an 18-item version of this scale in a
50
51 sample of nurses.
52
53 Holden et al. (2012) devised the Research Capacity and Culture Tool for use with
54
55 health professionals to measure indicators of research capacity at the individual, team, and
56
57
58
59
4
2
3 organisation levels. This scale consists of a series of statements, which participants rate on a
4
5
scale of 1-10. Sample items: “Find relevant literature” (individual level), “Provide resources
6
7
8 to support staff research training” (team), and “Has a plan or policy for research
9
10 development” (organisational).
11
12 Previous studies also have demonstrated that the research environment is critical for
13
14 advancing graduate student productivity (Royalty, Gelso, Mallinckrodt, & Garrett, 1986); for
15
16 example, the research training environment is an important factor in the training of graduate
17
18 students in applied areas in psychology (Kahn & Gelso, 1997). Gelso (1979) suggested that
19
Fo
20
21
there were 10 ingredients in the research training environment (i.e., all factors in graduate
22
23 training programs, departments, and universities that reflect positive attitudes toward
rP
24
25 research), including modelling appropriate research behaviours and attitudes, positive
26
ee
27 reinforcement and support for research efforts, early and minimally threatening involvement
28
29 in research, decoupling research and statistics, facilitating inward reflection for research
rR
30
31 ideas, emphasizing that science can be a partly social experience, teaching that all research is
32
ev
33
flawed and limited in some way, teaching varied investigative approaches to research,
34
35
36 teaching how science and clinical practice can be wedded, and focusing on how scholarly
iew
37
38 activities can be accomplished in practice settings.
39
40 Royalty et al. (1986) devised the 45-item Research Training Environment Scale
41
42 (RTES) to measure nine of the 10 ingredients of the research training environment that Gelso
43
44 (1979) suggested to be influential in promoting students’ research interest. The 10th
45
46
ingredient (i.e., focusing on how scholarly activities can be accomplished in practice settings)
47
48
49 was omitted, as this was considered an advanced skill. Then, Gelso (1993) abandoned the
50
51 idea of “decoupling research and statistics” as there was no empirical support for this aspect.
52
53 Gelso, Mallinckrodt, and Judge (1996) reformulated this concept as “teaching relevant
54
55 statistics and the logic of research design”, and included this in their revision, the 54-item
56
57
58
59
5
2
3 Research Training Environment Scale-Revised (RTES-R). A sample item: “My advisor
4
5
offers much encouragement to me for my research activities and accomplishments”.
6
7
8 Kahn and Gelso (1997) introduced the idea of instructional and interpersonal factors
9
10 within the research training environment. Interpersonal factors include early involvement,
11
12 faculty modelling, positive reinforcement, and research as a social experience; and
13
14 instructional factors include that all experiments are flawed, looking inward, teaching
15
16 relevant statistics, varied investigative styles, and wedding of science and practice. Kahn and
17
18 Miller (2000) created a short, 18-item form of the RTES-R by selecting one positively
19
Fo
20
21
worded item and one negatively worded item from each subscale based on corrected item-
22
23 total correlations.
rP
24
25 Present Study
26
ee
27 We employed a standard, classic test development approach (cf. DeVellis, 2016) to

28
29 develop and provide initial validation support for the PRE Scale for academics. We
rR
30
31 conducted a literature review to determine the underlying domains for the construct, and hold
32
ev
33
focus group discussions with higher education academics to validate these domains and
34
35
36 ensure that the items developed would be specific to the population’s experience. Next, we
iew
37
38 generated a list of approximately 70 items, had a group of experts rate them to examine their
39
40 content validity, and administered the items along with supporting validity scales to a large
41
42 sample of university-based academics. We used item analysis and exploratory factor analysis
43
44 (EFA) on one half of the data to trim the number of items to 25 and determine the factor
45
46
structure, and applied confirmatory factor analyses (CFA) on the second half of the data to
47
48
49 confirm the factor structure. The reliability and initial validity were then assessed.
50
51 Phase 1 - Item Development
52
53 The aim of this phase was to create sufficient items to form the basis for the new scale,
54
55 which was anticipated, for practical research purposes, to be approximately 25 items in
56
57
58
59
6
2
3 length. We originated approximately twice as many items as would appear in the final
4
5
measure (Hinkin, 1998; Kline, 2000). Items were generated based on a review of the
6
7
8 literature (e.g., Gelso, 1979) and a series of six focus groups conducted by the first, third, and
9
10 fourth authors, to engage with the target participants, validate the underlying domains of the
11
12 construct identified in the literature review, and enhance content validity of the scale items
13
14 (Vogt, King, & King, 2004). Participants were 42 academics (7 per group) from a state
15
16 university in Central Java, Indonesia, who were requested to reflect on and discuss their own
17
18 experiences and to give their opinions regarding characteristics of their university
19
Fo
20
21
environment that might motivate or impede them in the conduct of research-related activities.
22
23 We recorded the focus groups for later reference.
rP
24
25 Accordingly, we identified five broad domains of PRE salient to higher education
26
ee
27 academics: beneficial social relationships, positive reinforcement, support and expectations,

28
29 focus on research, and positive role models; and created 70 positively worded items (i.e.,
rR
30
31 positively worded to minimize response bias; Salazar, 2015), which were written in English
32
ev
33
to represent these five domains. Four independent reviewers rated the suitability of each item
34
35
36 to reflect a specific dimension of the construct. We removed 10 items that were considered
iew
37
38 overlapping or irrelevant, and adjusted several others, leaving a final list of 60 items.
39
40 Following Ægisdóttir, Gerstein, and Cinnarbas (2008), the first author translated the
41
42 items into Bahasa Indonesia, and two Indonesian academics examined the expression. The
43
44 items were then back-translated blindly into English by two bilingual Indonesian academics.
45
46
All authors compared the back-translated versions with the originals, and adjusted the
47
48
49 inaccuracies. The final items were then piloted with five Indonesian academics to check for
50
51 readability.
52
53 Phase 2 - Item Analysis and EFA
54
55
56
57
58
59
7
2
3 The aim of this phase was to identify items to be retained in the scale and to determine
4
5
the final factor structure. We used item and EFA procedures.
6
7
8 Method
9
10 Participants. We obtained data from 598 academics who were recruited from four
11
12 universities in Central Java, Indonesia, and divided this sample into two sub-samples using a
13
14 random split procedure to allow for a cross-validation test of results and to reduce sample-
15
16 specific effects that could potentially influence reliability and validity (Byrne, 2010). Sample
17
18 A from the split (N = 306) was used for Phase 2, and Sample B (N = 292) was used for Phase
19
Fo
20
21
3 and Phase 4.
22
23 Sample A consisted of 50.3% female academics (3.9% did not report gender), whose
rP
24
25 mean age was 42.29 years (SD = 10.12; 52.3% did not report age). A large majority (69.3%)
26
ee
27 had a masters’ degree, and 21.6% a doctorate (9.2% did not report education level). Only a
28
29 small proportion (1.3%) were professors, 22.5% associate professors, 27.8% assistant
rR
30
31 professors, 16% lecturers, and 4.6% junior lecturers (27.8% did not report position). The
32
ev
33
mean tenure was 16.57 years (SD = 10.04; 12.7% did not report tenure).
34
35
36 Sample B consisted of 54.1% female academics (3.4% did not report gender). The
iew
37
38 mean age was 43.39 years (SD = 9.74; 49% did not report age), most (62.7%) had a masters’
39
40 degree and 29.5% a doctorate (7.9% did not report education level). One percent were
41
42 professors, 20.5% associate professors, 35.6% assistant professors, 15.4% lecturers, and 3.1%
43
44 junior lecturers (24.3% did not report position). Mean tenure was 16.68 years (SD = 9.67;
45
46
13% did not report tenure).
47
48
49 There were no differences between Sample A and Sample B on any of the demographic
50
51 variables: age, t(1060) = 0.46, p = .65, gender, χ2(1) = 0.82, p = .37, tenure, χ2(2) = 6.93, p =
52
53 .06, and level of education χ2(2) = 0.20, p = .65.
54
55
Materials
56
57
58
59
8
2
3 The 60 items generated in Phase 1 which were expected to reflect the five domains of
4
5
PRE were administered in a questionnaire along with Organisational Culture/Support for
6
7
8 Research (OCSR) and Research Involvement (RI) Scales, which were used to test for
9
10 construct validity. Higher scores reflect higher levels of each construct.
11
12 PRE. This was assessed using the 60 items generated in Phase 1. Example items: “At
13
14 my university, academics often informally discuss research ideas in their day-to-day
15
16 discussions” (beneficial social relationship), “At my university, successful researchers have
17
18 high status” (positive reinforcement), “My university assists researchers to publish by helping
19
Fo
20
21
them with manuscript preparation (e.g., writing workshops)” (support and expectations), “At
22
23 my university, academics thinks research is important” (focus on research), and “At my
rP
24
25 university, many academics publish their research in high quality academic journals”
26
ee
27 (positive role models); 6-point-scale of 1 = strongly disagree to 6 = strongly agree.

28
29 Organisational culture/support for research. We used the 20-item OCSR Scale
rR
30
31 (Kortlik, Bartlett, Higgins, & Williams, 2002) to assess academics’ perceptions of the
32
ev
33
organisational culture or support for conducting research. Sample item: “My peers support
34
35
36 my efforts to publish in refereed research journals”; 6-point scale of 1 = strongly disagree to
iew
37
38 6 = strongly agree. Cronbach alpha was reported as .88, and validity was supported by
39
40 finding positive correlations with scales of research confidence (Kortlik et al., 2012).
41
42 Research involvement. We used the 24-item Research Involvement (RI) Scale
43
44 (Whelan et al., 2003) to assess participants’ level of engagement in research. Sample item:
45
46
“Participating in research as part of a collaborative team”; 6-point scale from 1 = not at all to
47
48
49 6 = a great deal. Alpha has been reported as .98, and construct validity has been supported by
50
51 finding positive correlations with evidence of greater research output (Whelan et al., 2013).
52
53 Procedure
54
55
56
57
58
59
9
2
3 The survey was administered by the chief researchers and assistants during working
4
5
hours. This study was conducted with approval from the authors’ university ethics committee,
6
7
8 and participants gave their permission to participate.
9
10 Results
11
12 Item analysis. We assessed: item skew to identify any item whose distribution
13
14 indicated floor or ceiling effects; the inter-item correlations to identify any pairs of items that
15
16 were too highly correlated (r ≥ .80), which might indicate that the items were redundant; the
17
18 corrected item-total correlations to identify any items with a weak or negative correlation
19
Fo
20
21
with the total scale (r < .30), which might indicate items that were not tapping the construct
22
23 of PRE; and age, gender, position, tenure, and level of education in relation to each item to
rP
24
25 identify items that might be responded to differently depending on demographic variables
26
ee
27 (Kline, 2000). No items were identified as problematic; thus, no items were eliminated.
28
29 EFA. The Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy (.94) and
rR
30
31 statistically significant Bartlett’s test of sphericity (p < .001) confirmed that the 60 items were
32
ev
33
suitable for factor analysis. The common variance is of interest in determining the underlying
34
35
36 factor structure, thus, we used common factor analysis (i.e., principal-axis factor analysis;
iew
37
38 Hair, Black, Babin, & Anderson, 2010). As the five anticipated factors were expected to be
39
40 correlated aspects of an overall perceived environment measure, we utilised a direct oblimin
41
42 rotation (Hair et al., 2010). Following Hayton, Allen, and Scarpello (2004), we used several
43
44 criteria to determine the number of factors: the scree plot, Velicer’s minimum average partial
45
46
test, parallel analysis (O’Connor, 2000), at least three items loading on a factor (Costello &
47
48
49 Osborne, 2005), and factor interpretability (Hinkin, 1998).
50
51 In the first EFA, the scree plot indicated five factors with eigenvalues > 1.0. Velicer’s
52
53 minimum average partial test and the parallel analysis also suggested a five-factor solution.
54
55 We accepted this solution as these five item groupings were interpretable theoretically and
56
57
58
59
10
2
3 reflected the five domains initially identified. Then, 35 items were deleted from the solution
4
5
because they did not load meaningfully on any factor, or the factor loadings were < .4 and/or
6
7
8 less than twice as strong on the appropriate factor as on the other factor (Hinkin, 1998). The
9
10 final five-factor solution accounted for 65.58% of the variance: Factor 1 = 32.94%, factor 2 =
11
12 15.56%, factor 3 = 6.81%, factor 4 = 5.40%, and factor 5 = 4.86%. Table 1 displays factor
13
14 loadings and eigenvalues.
15
16 Factor 1 (5 items; “beneficial social relationship”) includes the situation where
17
18 individuals perceive that their social relationships support their participation in research-
19
Fo
20
21
related activities (α = .88, M = 23.01, SD = 3.85). Factor 2 (5 items; “positive reinforcement”)
22
23 refers to the situation where individuals perceive positive social and non-social reinforcement
rP
24
25 for achievements in research-related activities (α = .92, M = 20.62, SD = 5.90). Factor 3 (5
26
ee
27 items; “support and expectations”) reflects perceived support and expectations from the
28
29 university for conducting research-related activities (α = .86, M = 24.85, SD = 3.21). Factor 4
rR
30
31 (5 items; “focus on research”) includes perceived emphasises on research-related activities (α
32
ev
33
= .92, M = 22.69, SD = 4.53). Factor 5 (5 items; “positive role models”) reflects perceived
34
35
36 availability of positive role model for conducting research-related activities (α = .89, M =
iew
37
38 21.81, SD = 4.36). The associations among the five factors (range .10 to .47; all p < .001)
39
40 were in line with the results from the EFA indicating that the subscales were somewhat
41
42 independent, but with overlap among them. Alpha for the full scale was .92.
43
44 Insert Table 1 about here
45
46
Phase 3 - CFA
47
48
49 This phase aimed to validate the factor structure of the PRE Scale with Sample B.
50
51 Using CFA (AMOS Version 4.0; Arbuckle & Wothke, 1995). We examined the 5-factor
52
53 structure identified in Phase 2, and then compared this model with a 1-factor model, a
54
55 hierarchical, 2nd-order model, and a bifactor model (Reise, Bonifay, & Haviland, 2013).
56
57
58
59
11
2
3 We used the χ2 statistic, the normed χ2 (χ2/df), the Comparative Fit Index (CFI),
4
5
Tucker-Lewis Index (TLI), and the Root Mean Square Error of Approximation (RMSEA) to
6
7
8 examine model fit. A significant χ2, χ2/df < 3.0, CFI and TLI values > .92, and RMSEA < .08
9
10 indicate satisfactory fit when sample size < 250 and observed variables number between 12
11
12 and 30. We then compared the different models using the χ2-difference test and the Akaike
13
14 Information Criterion (AIC), where the lower value indicates a better fit (Hair e al., 2010).
15
16 The 5-factor model identified in Phase 2 generated satisfactory fit statistics (see Table
17
18 2 for fit statistics for all models). All factor loadings were statistically significant (p < .001)
19
Fo
20
21
and ranged from .72 to .84 (beneficial relationship), .83 to .91 (positive reinforcement), .55 to
22
23 .89 (support and expectations), .76 to .89 (focus on research), and .70 to .89 (positive role
rP
24
25 models). Correlations among the latent variables ranged from .10 to .62.
26
ee
27 The 2nd-order model (correlations with 2nd-order factor = .43 to .87) and the bifactor
28
29 model, but not the 1-factor model, also had satisfactory fit statistics. However, the best-fitting
rR
30
31 model was the bifactor model, which was statistically different from the 5-factor model, and
32
ev
33
generated the lowest AIC. The bifactor model included a general latent variable (i.e.,
34
35
36 dependent on all 25 items) and five subscale latent variables (i.e., five uncorrelated factors
iew
37
38 dependent on their respective five items). This model showed that each item is an indicator of
39
40 both a total and subscale aspect, with the results for the total variable representing common
41
42 sources of variance after controlling for subscale variances, and the subscale variables
43
44 representing variances after controlling for the total variance (Reise et al., 2013).
45
46
Insert Table 2 about here
47
48
49 As our results supported multi-dimensionality of the scale (i.e., the 5-factor, 2nd-order,
50
51 and bifactor models all had acceptable fit statistics), it needed to be determined whether the
52
53 scale should be interpreted at the global or subscale level. We calculated bifactor model-
54
55 based reliability estimates and compared the variances explained for the total and subscale
56
57
58
59
12
2
3 interpretations. Following Rodriguez et al. (2016), we used the Bifactor Indices Calculator
4
5
(Dueber, 2017) to calculate Omega, OmegaH, Relative Omega, and the explained common
6
7
8 variance (ECV). Omega, the estimate of the proportion of variance accounted for when
9
10 considering all items in a factor, was .96 (general factor), and .91, .94, 86, .92, and .90
11
12 (specific factors), indicating high reliability for the general factor and sound to high reliability
13
14 for the specific factors, and suggesting all factors have acceptable reliability. OmegaH, the
15
16 proportion of unique variance explained by a factor, was .71 (general factor), and was .37,
17
18 .62, and .52, .62, and .25 (specific factors). Relative Omega, the proportion of reliable
19
Fo
20
21
variance in the multidimensional composite due to a factor, was .74 (general factors), and
22
23 was .41, .68, .60, .67 and .28 (specific factors), indicating the majority of reliable variance in
rP
24
25 the total scores resides within the general factor. Last, the ECV, the proportion of all common
26
ee
27 variance explained by a factor, was .40 (general factors), and was .09, .21, .10, .14, and .06
28
29 (specific factors), suggesting a moderately strong global factor, with much less variance
rR
30
31 explained by the specific factors. Thus, it can be concluded that interpretation at the total
32
ev
33
level (and not the subscale level) will give the more useful measure of PRE, as the total factor
34
35
36 will account for more meaningful levels of variance in the construct.
iew
37
38 Phase 4: Construct Validity
39
40 This phase aimed to evaluate the initial construct validity of the scale by correlating
41
42 scores from the PRE Scale with scores from OCSR and RI Scales. Individuals who reported
43
44 stronger research training environments were more likely to perceive their environment to be
45
46
more research supportive (Kahn & Miller, 2000), and those who perceived a more research
47
48
49 supportive work environment were more likely to report a positive attitude towards research
50
51 and greater research engagement (Royalty et al., 1986). Thus, we expected PRE Scale to be
52
53 associated positively with OCSR and RI Scales. These analyses were conducted using
54
55 Sample B (N = 292). All correlations were statistically significant and in the expected
56
57
58
59
13
2
3 directions, as reported in Table 3. The results demonstrated that the PRE Scale was associated
4
5
with the two other constructs as expected; supporting convergent validity of the scale.
6
7
8 Insert Table 3 about here
9
10 Discussion
11
12 We devised and reported initial validity evidence for a psychometrically sound, 25-
13
14 item scale to assess PRE for higher education academics: The Perceived Research
15
16 Environment (PRE) Scale. We operationalised PRE as the organisational environment
17
18 perceived by academics, which enables and supports them to learn and conduct research-
19
Fo
20
21
related activities and yield scientific publications. The salient domains identified were
22
23 beneficial social relationships, positive reinforcement, support and expectations, focus on
rP
24
25 research, and positive role models. Content validity was supported by a review of the
26
ee
27 literature, focus groups, pilot testing, and use of expert reviewers, whereas construct validity
28
29 was supported by the EFAs and CFAs, which indicated that the new scale reflected the five
rR
30
31 anticipated inter-correlated domains. The measure can most productively be applied at the
32
ev
33
global level, that it is internally reliable, and that the positive associations with the OCSR and
34
35
36 the RI Scale supported its convergent validity.
iew
37
38 Previous research has shown the importance of PRE in academics (e.g., Duffy et al.,
39
40 2013). The present study provided a comprehensive measure of PRE, which assesses various
41
42 aspects of the PRE construct. At 25 items, the scale will be practical and suitable to be used
43
44 simultaneously with other scales in both research and practice. Extending PRE research using
45
46
this scale has the potential to add to our knowledge and understanding of the PRE from the
47
48
49 perspective of higher education academics, for example when designing research studies,
50
51 identifying its nomological network, and examining its across-time correlates.
52
53 The PRE Scale also will be of use to those who work with academics at any stage of
54
55 their career when they have issues with research-related performance, and it will be useful to
56
57
58
59
14
2
3 help explore helps and blockages related to their research-related progress and achievement.
4
5
Practitioners can use the scale as a diagnostic tool at an early stage of career counselling, as
6
7
8 well as an evaluation instrument after a series of counselling sessions. At the organisational
9
10 level, the scale can be used for human resource mapping as a foundation for formulating
11
12 policies at department, research centre, and university levels.
13
14 Limitations
15
16 Our study was conducted using a sample of higher education academics from several
17
18 universities in Central Java. Hence, the conclusions of this study need to be tested on other
19
Fo
20
21
academic populations. We examined content and construct validity of the scale, and future
22
23 researchers could focus on establishing predictive validity by, for example, testing the
rP
24
25 longitudinal associations between scores on the scale at one point in time and subsequent
26
ee
27 outcomes. We showed that the scale was unrelated to several demographic variables (e.g.,
28
29 age, gender, tenure, level of education), suggesting no inherent bias based on these
rR
30
31 characteristics; however, future studies need to examine structural invariance on these and
32
ev
33
other variables to support these results.
34
35
36 Conclusion
iew
37
38 The current research has demonstrated support for a scale to measure PRE for higher
39
40 education academics, although future studies are needed to extend support for its validity and
41
42 to test its applicability on more diverse populations. Our findings contribute to the body of
43
44 literature on academics’ PRE, and open the way for improved career counselling for
45
46
academics, research development interventions, and organisational policies.
47
48
49
50
51
52
53
54
55
56
57
58
59
15
2
3 References
4
5
Ægisdóttir, S., Gerstein, L, H., & Cinnarbas, C. D. (2008). Methodological issues in cross-
6
7
8 cultural counselling research: Equivalence, bias, and translations. The Counseling
9
10 Psychologist, 36, 188-219. doi:10.1177/0011000007305384
11
12 Arbuckle, J. L. & Wothke, W. (1995). Amos 4.0 user’s guide. Chicago, IL: Small Waters
13
14 Corporation.
15
16 Bland, C. J., & Ruffin, M. T. (1992). Characteristics of a productive research environment:
17
18 Literature review. Academic Medicine, 67, 385-397. doi:10.1097/00001888-
19
Fo
20
21
199206000-000-10
22
23 Bronfenbrenner, U. (1979). The ecology of human development: Experiments by nature and
rP
24
25 design. Cambridge, MA: Harvard University Press.
26
ee
27 Byrne, B. (2010). Structural equation modelling with AMOS: Basic concepts, applications,
28
29 and programming. New York, NY: Routledge.
rR
30
31 Chen, Y., Gupta, A., & Hoshower, L. (2006). Factors that motivate business faculty to
32
ev
33
conduct research: An expectancy theory analysis. Journal of Education for Business,
34
35
36 81, 179-189. doi:10.3200/JOEB.81.4.179-189
iew
37
38 Costello, A. B., & Osborne, J. W. (2005). Best practices in exploratory factor analysis: Four
39
40 recommendations for getting the most from your analysis. Practical Assessment,
41
42 Research & Evaluation, 10, 1-9. Retrieved from http://pareonline.net/pdf/v10n7.pdf
43
44 DeVellis, R. F. (2016). Scale development: Theory and applications (Vol. 26). Los Angeles,
45
46
CA: Sage.
47
48
49 Dueber, D. M. (2017). Bifactor Indices Calculator: A Microsoft Excel-based tool to calculate
50
51 various indices relevant to bifactor CFA models.
52
53 https://dx.doi.org/10.13023/edp.tool.01 [Available at
54
55 http://sites.education.uky.edu/apslab/resources/]
56
57
58
59
16
2
3 Duffy, R. D., Torrey, C. L., Bott, E. M., Allan, B. A., & Schlosser, L. Z. (2013). Time
4
5
management, passion, and collaboration: A qualitative study of highly research
6
7
8 productive counseling psychologists. The Counseling Psychologist, 41, 881-917. doi:
9
10 10.1177/0011000012457994
11
12 Eam, P. (2015). Investigating relationship among research self-efficacy, research outcome
13
14 expectations, and research interest of Cambodian faculty: Testing social-cognitive
15
16 theory. International Journal of Sociology of Education, 4, 199-224.
17
18 doi:10.17583/rise.2015.1752
19
Fo
20
21
Gelso, C. J. (1979). Research in counseling. Methodological and professional issues. The
22
23 Counseling Psychologist, 8, 7-35.
rP
24
25 Gelso, C. J. (1993). On the making of a scientist-practitioner: A theory of research training in
26
ee
27 professional psychology. Professional Psychology: Research and Practice, 24, 468-

28
29 476. doi:10.1037/0735-7028.24.4.468
rR
30
31 Gelso, C. J., Mallinckrodt, B., & Judge, A. B. (1996). Research training environment, attitude
32
ev
33
toward research, and research self-efficacy. The revised Research Training
34
35
36 Environment Scale. The Counseling Psychologist, 24, 304-322.
iew
37
38 doi:10.1177/0011000096242010
39
40 Hair, J. F., Black, W. C., Babin, B. J., Anderson, R. E. (2010). Multivariate data analysis. A
41
42 global perspective. (7th ed.). New Jersey, NJ: Prentice Hall.
43
44 Hayton, J. C., Allen, D. G., & Scarpello, V. (2004). Factor retention decisions in exploratory
45
46
factor analysis: A tutorial on parallel analysis. Organizational Research Methods, 7,
47
48
49 191-205. doi:10.1177/1094428104263675
50
51 Hinkin, T. R. (1998). A brief tutorial on the development of measures for the use in survey
52
53 questionnaires. Organizational Research Methods, 1, 104-121.
54
55 doi:10.1177/109442819800100106
56
57
58
59
17
2
3 Holden, L., Pager, S., Golenko, X., & Ware, R. S. (2012). Validation of the Research
4
5
Capacity and Culture (RCC) tool: Measuring RCC at individual, team, and organisation
6
7
8 levels. Australian Journal of Primary Health, 18, 62-67. doi:10.1071/PY10081
9
10 Kahn, J. H., & Gelso, C. J. (1997). Factor structure of the Research Training Environment
11
12 Scale – Revised: Implications for research training in applied psychology. The
13
14 Counseling Psychologist, 25, 22-37. doi:10.1177/0011000097251004
15
16 Kahn, J. H., & Miller, S. A. (2000). Measuring global perceptions of the Research Training
17
18 Environment Using A Short Form of the RTES-R. Measurement and Evaluation in
19
Fo
20
21
Counseling and Development, 33, 103-119. Retrieved from
22
23 https://search.proquest.com/docview/195607998?pq-origsite=gscholar
rP
24
25 Kortlik, J. W., Bartlett II, J. E., Higgins, C. C., & Williams, H. A. (2002). Factors associated
26
ee
27 with research productivity of agricultural education faculty. Journal of Agricultural

28
29 Education, 43, 1-10. doi:10.5032/jae.2002.03001
rR
30
31 Lindsay, R., Breen, R., & Jenkins, A. (2002). Academic research and teaching quality. The
32
ev
33
views of undergraduate and postgraduate students. Studies in Higher Education, 27, 309-
34
35
36 327. doi:10.1080/03075070220000699
iew
37
38 Mallinckrodt, B. (1997). Discovering the research training environments that fit the students
39
40 we select: The challenge of providing for diverse needs. The Counseling Psychologist, 25,
41
42 68-73. doi:10.1177/0011000097251006
43
44 Marsh, G. W., & Brown, T. L. (1992). The measurement of nurses’ attitudes towards nursing
45
46
research and the research environment in clinical settings. Journal of Clinical Nursing, 1,
47
48
49 315-322. doi:10.1111/j.1365-2702.1992.tb00425.x
50
51 Nguyen, Q., Klopper, C., & Smith, C. (2016). Affordances, barriers, and motivations:
52
53 Engagement in research activity by academics at the research-oriented university in
54
55
56
57
58
59
18
2
3 Vietnam. Open Review of Educational Research, 3, 68-84.
4
5
doi:10.1080/23265507.2016.1170627
6
7
8 Owen, M. (1992). Research at small Canadian universities. The Canadian Journal of Higher
9
10 Education, 22, 1-14. Retrieved from
11
12 http://journals.sfu.ca/cjhe/index.php/cjhe/article/view/183130/183106
13
14 Pranulis, M. F., & Gortner, S. R. (1985). Researchmanship: Characteristics of productive
15
16 research environments in nursing. Western Journal of Nursing Research, 7, 127-131.
17
18 doi:10.1177/00920055X8500700112
19
Fo
20
21
Reise, S. P., Bonifay, W. E., & Haviland, M. G. (2013). Scoring and modelling psychological
22
23 measures in the presence of multidimensionality. Journal of Personality Assessment, 95,
rP
24
25 129-140. doi:10180/00223891.2012.725437
26
ee
27 Rodriguez, A., Reise, S. P., & Haviland, M. G. (2016). Evaluating bifactor models:
28
29 Calculating and interpreting statistical indices. Psychological Methods, 21, 137-150.
rR
30
31 doi:10.1037/met0000045
32
ev
33
Royalty, G. M., Gelso, C. J., Mallinckrodt, B., & Garrett, K. (1986). The environment and the
34
35
36 student in counselling psychology: Does the research training environment influence
iew
37
38 graduate students’ attitudes toward research? The Counseling Psychologist, 14, 9-30.
39
40 doi:10.1177/0011000086141002
41
42 Salazar, M. (2015). The dilemma of combining positive and negative items in scales.
43
44 Psicothema, 27, 192-199. doi:10.7334/psicothema2014.266
45
46
Shivy, V. A., Worthington, E. L., Wallis Jr, A. B., & Hogan Jr, C. (2003). Doctoral research
47
48
49 training environments (RTEs): Implications for the teaching in psychology. Teaching of
50
51 Psychology, 30, 297-302. doi:10.1207/S1528023TOP3004_03
52
53
54
55
56
57
58
59
19
2
3 Vogt, D. S., King, D. W., & King, L. A. (2004). Focus groups in psychological assessment:
4
5
Enhancing content validity by consulting members of the target population.
6
7
8 Psychological Assessment, 16, 231-243. doi:10.1037/1040-3590.16.3.231
9
10 Whelan, K., & Markless, S. (2012). Factors that influence research involvement among
11
12 registered dietitians working as university faculty: A qualitative interview study. The
13
14 Academy of Nutrition and Dietetics, 112, 1021-1028. doi:10.1016/j.jand.2012.03.002
15
16 Whelan, K., Copeland, E., Oladitan, L., Murrells, T., & Gandy, J. (2013). Development and
17
18 validation of a questionnaire to measure research involvement among registered
19
Fo
20
21
dietitians. Journal of the Academy of Nutrition and Dietetics, 113, 563-568.
22
23 doi:10.1016/j.jand.2012.08.027
rP
24
25 White, C. S., James, K., Burke, L. A., & Allen, R. S. (2012). What makes a “research star”?
26
ee
27 Factors influencing research productivity of business faculty. International Journal of

28
29 Productivity and Performance Management, 61, 584-602.
rR
30
31 doi:10.1108/17410401211249175
32
ev
33
Young, K., & Rice, M. (1983). Measurement of attitude toward the nursing research
34
35
36 environment in the university setting. Paper presented at the First Annual Scientific
iew
37
38 Meeting on Research in Nursing Education, San Francisco, California.
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
1
2 20
3
4
5 Table 1
6 Factor Loadings for the PRE Scale; Sample A (N = 306)
7 Items Factor Factor Factor Factor Factor
8 1 2 3 4 5
9 1. At my university, academics are willing to involve their colleagues in research projects .86 -.10 -.04 -.07 -.01
10 2. At my university, academics support one another in their research projects .83 .11 .13 -.04 .01
11 3. At my university, academics often informally discuss research ideas in their day-to-day discussions .83 -.09 -.04 .03 -.01
12 4. There is a sense at my university that academics enjoy their research activities .71 .06 .04 .05 .22
13 5. At my university, researchers actively involve students in their research projects .70 -.09 -.01 .20 -.01
14 6. My university rewards successful researchers .03 -.90 -.01 .07 .01
15 7. My university is well known for its research expertise -.01 -.89 .01 -.03 .04
16 8. At my university, researchers who do well are highly respected by their colleagues .04 -.85 .09 -.01 -.02
Fo
17 9. At my university, there are incentives for successful research activities (e.g., getting published or obtaining a research grant -.01 -.83 .03 .11 -.09
18 10. At my university, we all celebrate when a colleague is successful (e.g., gets published or obtains a research grant .08 -.76 .13 -.09 .11
11. My university sets clear expectations regarding research output for academics .01 .01 .82 -.06 .01
rP
19
20 12. My university has specific programs and funds to help new academics get their research started .04 .01 .79 -.02 .05
21 13. My university assists researchers to publish by helping them with manuscript preparation (e.g., writing workshops) -.01 -.03 .78 .02 -.03
ee
22 14. My university has clear expectations that academics will engage in research
r .09 -.08 .76 .07 -.05
23 15. Academics at my university know it is expected of them that they attend conferences and present their research -.08 -.07 .68 .11 -.05
16. At my university, academics are encouraged to use a wide variety of research methods in their research -.01 .05 .06 -.91 -.01
24
17. At my university, researchers are always on the look-out for research collaborators .06 -.02 .04 -.86 .03
25
Re
18. At my university, academics thinks research is important .03 .02 .03 -.85 -.04
26
27
v
19. At my university, new faculty members are encouraged to publish as soon as they commence work -.05 -.14 -.08 -.79 .09
20. At my university, opportunity for academics to actualize themselves in research is widely open .08 .16 .08 -.78 .07
28
21. Academics at my university give high priority to their research .02 .04 .14 -.03 .86
29
iew
22. At my university, many academics publish their research in high quality academic journals .01 -.03 -.06 -.01 .86
30
23. Academic at my university are strongly focused on research -.13 .03 -.05 .09 .83
31 .13 .01 .10 -.01
24. Academics at my university strive to publish their research in high quality journals .77
32 .14 -.05 .01 .04 .70
25. Many academics at my university are working on important research projects
33 Eigenvalues 8.57 4.21 2.06 1.69 1.55
34 % variance explained 32.94 15.56 6.81 5.40 4.86
35 Note. Factor 1 = Beneficial social relationships, Factor 2 = Positive reinforcement, Factor 3 = Support, Factor 4 = Encouragement, Factor 5 = Role model; Main loadings
36 highlighted in bold.
37
38
39
40
41
42
43
44
46
47
1
2 21
4
5 Table 2
6
7 Model Fit Indices of the 3-Factor, 1-Factor, 2nd-Order Factor, and Bifactor Models for Sample B
8
9 (N = 292)
10
11 Model χ2 df χ2/df CFI TLI RMSEA χ2Diff AIC
12
5-factor 559.69*** 256 2.19 .94 .94 .06 - 697.69
13
14 1-factor 2324.78*** 270 8.61 .63 .59 .16 p < .001 2434.78
15
16 2nd order 749.03*** 269 2.78 .91 .90 .08 p < .001 861.03
Fo
17
18 Bifactor 513.33*** 244 2.10 .95 .94 .06 p < .001 675.33
rP
19
20 Note. χ2Diff statistics refer to differences with 5-factor model. *** p < .001
21
ee
22 r
23
24
25
Re
26
27
v
28
29
iew
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
46
47
1
2 22
4
5 Table 3
6
7 Summary Data for Sample B (N = 292; correlations above diagonal)
8
9 Indonesian
10
Scale M SD Range α 1 2 3 4 5 6 7 8
11
12 1. Full scale 112 16.65 28-145 .93 - .81** .65** .63** .80** .64** .47** .49**
13
14 2. Subscale 1 (beneficial social relationship) 22.87 4.29 5-30 .90 - .40** .48** .52** .54** .30** .35**
15
3. Subscale 2 (positive reinforcement) 20.36 6.22 5-30 .94 - .12* .49** .10* .14* .24**
16
Fo
17 4. Subscale 3 (support and expectations) 24.44 3.70 6-30 .87 - .38** .49** .38** .32**
18
rP
19 5. Subscale 4 (focus on research) 22.53 4.80 5-30 .91 - .39** .54** .58**
20
21 6. Subscale 5 (positive role models) 21.79 4.59 5-30 .89 - .35** .27**
ee
22 7. Organisational culture/ support for research 67.24 15.40 24-144
r .93 - .54**
23
24 8. Research involvement 100.82 21.12 18-108 .97 -
25
Re
26 Note. * p < .05; ** p < .01; *** p < .001 v
27
28
29
iew
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
46
47
2. Bukti Konfirmasi Review dan
Hasil Review Pertama
(7 Okt 2018)
8/1/2020 Gmail - Journal of Psychoeducational Assessment - Decision on Manuscript ID JPA-18-0139
Journal of Psychoeducational Assessment - Decision on Manuscript ID JPA-18-

0139
Journal of Psychoeducational Assessment <onbehalfof@manuscriptcentral.com> Sun, Oct 7, 2018 at 2:08 AM
Reply-To:
To:
Cc:
06-Oct-2018
Dear Dr. :
Manuscript ID JPA-18-0139 entitled "Development and Initial Validation of Perceived Research Environment Scale
for Higher Education Academics" which you submitted to Journal of Psychoeducational Assessment, has been
reviewed. The comments of the reviewer(s) are included at the bottom of this letter. I appreciate your patience with
the review process.
The reviewers noted strengths in the manuscript, particularly its methods and execution; however, the reviewers also
expressed concerns about it. Based on the reviewers’ responses, I ask that you revise your manuscript to address
their comments in a revision. The reviewers were helpful in detailing ways to think about these issues and adjust the
manuscript accordingly, so I encourage you to examine their comments carefully as you revise. It is likely I will send
your revised manuscript to one of these initial reviewers. As with any manuscript, I cannot guarantee that these efforts
will translate to acceptance in JPA.
To revise your manuscript, log into https://mc.manuscriptcentral.com/jopa and enter your Author Center, where you
will find your manuscript title listed under "Manuscripts with Decisions." Under "Actions," click on "Create a
Revision." Your manuscript number has been appended to denote a revision.
You may also click the below link to start the revision process (or continue the process if you have already started
your revision) for your manuscript. If you use the below link you will not be required to login to ScholarOne
Manuscripts.
*** PLEASE NOTE: This is a two-step process. After clicking on the link, you will be directed to a webpage to confirm.
***
https://mc.manuscriptcentral.com/jopa?URL_MASK=54cc667e9616460e98dd8a1ed9a595ed
You will be unable to make your revisions on the originally submitted version of the manuscript. Instead, revise your
manuscript using a word processing program and save it on your computer.
Once the revised manuscript is prepared, you can upload it and submit it through your Author Center.
When submitting your revised manuscript, you will be able to respond to the comments made by the reviewer(s) in
the space provided. You can use this space to document any changes you make to the original manuscript. In order
to expedite the processing of the revised manuscript, please be as specific as possible in your response to the
reviewer(s).
IMPORTANT: Your original files are available to you when you upload your revised manuscript. Please delete any
redundant files before completing the submission.
Because we are trying to facilitate timely publication of manuscripts submitted to Journal of Psychoeducational
Assessment, your revised manuscript should be submitted within 30 days from receipt of this letter. If it is not possible
for you to submit your revision in this amount of time, please advise the Associate Editor before the 30 day period as
the link to you article will expire and you will not be able to re-submit your paper without making a specific request.
Once again, thank you for submitting your manuscript to Journal of Psychoeducational Assessment and I look
forward to receiving your revision.
Sincerely,
Dr. Renée Tobin
Associate Editor, Journal of Psychoeducational Assessment
tobin@temple.edu
Don Saklofske
Editor
Reviewer(s)' Comments to Author:
Reviewer: 1
Comments to the Author

Thank you for the opportunity to review this manuscript (JPA-18-0139). This article described the development and
validation of the Perceived Research Environment Scale (PRES), a faculty-report measure for use in higher
education institutions. I hope the following comments will assist the authors in revising their manuscript.
1.) This study seems to lack of a cohesive, unifying theoretical framework. The authors vaguely reference potential
applications of Bronfenbrenner’s ecological theory, but it is unclear how this theory guided instrument development
and the analyses. The authors also reference several studies exploring various aspects of research environments,
but these variables do not seem to be united in any particular framework. It would have been more compelling for the
authors to apply a specific theoretical framework to guide the development of the measure.
2.) On page 7, the authors describe a method for randomly assigning participants to Samples A and B. They
contend that the groups were comparable with respect to several demographic variables; however, I am most
interested in knowing whether the groups were comparable with respect to institution. Participants came from only
four institutions, and it is likely that participants from the same university would rate their research environments
similarly (given that they are likely experiencing similar pressures). Is it possible that either Samples A or B consisted
of a disproportionately large or small number of faculty from any one of the four institutions? This might impact the
results. Ideally, the sample would have comprised faculty from a variety of different institutions.
On a related note, it might be interesting to see whether faculty from the same institution had similar ratings of their
research environments. I suppose the ratings might be similar for some items (availability of funding) but less similar
for items that vary more across individuals (e.g., access to informal mentoring).
3.) The authors note that there were no significant differences between Sample A and Sample B with respect to
age. I’m not sure how meaningful this assertion is, given that approximately 50% of participants in both samples did
not report their ages. This constitutes a fairly large amount of missing data.
In general, how were missing data handled in this study? This is important to discuss in the manuscript.
4.) I would have liked to have had some more information about the 42 academics included in the focus groups as
well as the four independent reviewers who rated the suitability of the items (p. 6). For example, who were the
reviewers and what qualified them for this task?
5.) The authors administered a measure of research involvement (i.e., the Research Involvement Scale). I would
be curious to know if Samples A and B differed with respect to scores on this measure. If one group were more
involved in research than the other, the two samples would not be comparable on a very important dimension
(especially given the nature of the instrument the authors are developing).
6.) The authors sampled participants with a wide range of academic roles, including professors and lecturers. I can
imagine that some of these positions are not inherently or contractually research roles. Would respondents with
positions that were non-research oriented be the best respondents for this type of measure?
7.) The sample included very few full professors (i.e., approximately 1% of each subsample), which seems
problematic for a couple of reasons. First, how might this have impacted the authors attempts to determine whether
items were responded to differently by faculty in various positions? Were subsamples for each position (e.g., full
professor, associate professor, assistant professor) large enough (and comparable enough in size) to detect response
differences across groups? Second, full professors may have more institutional knowledge and research experience
than associate and assistant professors. To have so few in the sample appears to be a notable limitation of this study.
8.) I would have also recommended that the authors collect data regarding the length of time participants had been
at their respective institutions. Newer faculty might not have had enough time to form opinions about their respective
research environments.
9.) I would have been interested to see the internal consistency values for the PRES total score and subscale
scores in Sample B. I would recommend the authors report these values.
10.) For the CFA, the authors suggest that three of the four models (i.e., 2nd order model, 5-factor model, and
bifactor model) had satisfactory fit statistics. However, CFI values for the 5-factor and 2nd order models were below
.95. Some research has suggested that a more appropriate criterion for CFI values is .95 or greater (rather than .90
or greater). The authors may wish to revise their language accordingly. (Please see Hu and Bentler, 1999; reference
provided below.)
11.) Minor comments:

a. On page 3, the authors write, “Duffy et al. (2013) interview 17 of the most research-productive counselling
psychologists within the American Psychological Association accredited counselling program.” Did the authors mean
17 faculty across a variety of APA-accredited programs?
b. In the implications section, I would further emphasize the potential value of the PRES for informing
organizational change. This point is a good one and should be further developed. For example, more detail about the
specific uses of the instrument for facilitating organizational improvement would be interesting (and would ultimately
make the paper more compelling).
Reference
Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis. Conventional criteria
versus new alternatives. Structural Equation Modeling, 6, 1-55.
Reviewer: 2

This manuscript reports on the development and psychometric evolution of a measure of perceived research
environment. Specifically, the measure is designed to tap into a construct comprised of a number of dimensions
related to an individual’s perception of the availability of resources, support, and appreciation of research efforts
within the institution in which the individual is employed. I found the manuscript to be clear and well-written. The
authors did a nice job explaining the concept of the perceived research environment and why it is important, as well
as the previous attempts to at psychometric instruments to capture the construct. As illustrated in the introduction,
previous measures have been quite specific to either certain fields (e.g., nursing), or populations (e.g., graduate
students). Thus, the authors make a good case for a general perceived research environment scale that could be
applicable to Universities or other research institutions, regardless of the specific discipline. However, I would
recommend the authors add a paragraph at the end of the introduction, before the “Present Study” section,
summarizing this and making this rationale more explicit.
There are other strengths of this paper, including conducting item analyses, evaluation of the internal structure with
EFA followed by a CFA in a random hold-out sample to confirm the factor structure. The factor analytic methods were
appropriate and fit the theoretical conception of the construct, including use of principal axis factoring, direct oblimin
rotation, which allows for correlations among the rotated factors, and the use of Velicer’s MAP and parallel analysis to
inform the decision on the number of factors to retain. With that said, there are also some areas that should be
addressed to improve the paper and the contribution of the study.
I appreciate the fact that the authors conducted focus groups as one of the methods for identifying the important
domains of the construct. It would probably be useful if the authors provided a bit more detail about how the
information from the focus groups was analyzed, and what dimensions they identified, independent of the dimensions
that the identified from the literature review.
The conduct of the item analyses, including evaluation of item response distributions, item-total correlations, and
inter-item correlations was appropriate, and an often over-looked step in the process of test development. Although
the authors indicated that none of the items were eliminated based upon the item analyses, it would still be useful
information to present the results of the item analyses in a table, including mean, standard deviation, skewness,
kurtosis, and item-total correlation of each item, as well as the mean item-total correlation, if not for the original set of
items, at least for the final set of items after eliminating items due to factor loading issues.
For the EFA, the authors indicate that both the minimum average partial (MAP) and parallel analysis procedures
indicated 5 factors, but they do not present the results of the analysis. It is good practice to provide the numbers
generated from these procedures that lead to the conclusion. So, for the parallel analysis, the first 6 actual and
random eigenvalues can be presented.
Regarding the CFA, the procedures were reasonable, and the fit statistics were appropriate. The computation of
Omega’s was also useful. My one suggestion here would be to present a table or figure showing the loadings of the
bifactor model, which was determined to be the best fitting model.
The validity analyses were fairly limited in scope, but the measures used were reasonable and the correlations found
were supportive of construct validity.
The Discussion section was a bit lacking in content. Given that construct validation requires a multitude of evidence
from different methods and perspectives, what is especially needed in the discussion section is suggestions for
further developments, such as other constructs, measures, and criterion variables would be useful to further establish
the nomological network and construct validity of scores from this measure. I found the statement that the measure
would be useful for “early stages of career counseling” to be perplexing. Perhaps I am missing something, but it
seems this measure would only be applicable to someone who is already working in an institution with some kind of
research focus. Otherwise, what “research environment” are they reporting on. In addition, it is premature to
recommend a measure for applied use after one development study.
3. Bukti Konfirmasi Submit Revisi Pertama,
Respon kepada Reviewer,
dan Artikel yang Diresubmit
(3 Nov 2018)
8/1/2020 Gmail - Journal of Psychoeducational Assessment - Manuscript ID JPA-18-0139.R1
Journal of Psychoeducational Assessment - Manuscript ID JPA-18-0139.R1

Journal of Psychoeducational Assessment <onbehalfof@manuscriptcentral.com> Sat, Nov 3, 2018 at 8:08 PM
To:
03-Nov-2018
Dear Dr. :
Your manuscript entitled "Development and Initial Validation of Perceived Research Environment Scale
for Higher Education Academics" has been successfully submitted online and is presently being given full
consideration for publication in Journal of Psychoeducational Assessment.
Your manuscript ID is JPA-18-0139.R1.
Sincerely,
RESPONSE TO THE REVIEWERS
OF THE JOURNAL OF PSYCHOEDUCATIONAL ASSESSMENT
MANUSCRIPT ID: JPA – 18 – 0139
Reviewer: 1

Thank you for the opportunity to review this manuscript (JPA-18-0139). This article
described the development and validation of the Perceived Research Environment Scale
(PRES), a faculty-report measure for use in higher education institutions. I hope the
following comments will assist the authors in revising their manuscript.
1.) This study seems to lack of a cohesive, unifying theoretical framework. The authors
vaguely reference potential applications of Bronfenbrenner’s ecological theory, but it is
unclear how this theory guided instrument development and the analyses. The authors also
reference several studies exploring various aspects of research environments, but these
variables do not seem to be united in any particular framework. It would have been more
compelling for the authors to apply a specific theoretical framework to guide the
development of the measure.
Response:
We updated our previous explanations regarding Bronfenbrenner’s theory and this section
on pages 2-3:
The important role of the research environment for academics is consistent with
Bronfenbrenner’s (1979, 1986) ecological systems theory. This perspective emphasises that,
compared to the objective environment, perceptions of the environment are of primary
significance, because it is these that affect and guide behaviour. Bronfenbrenner argued that
developing individuals are surrounded by interrelated systems. The inner circle, or
microsystem, is where academics have direct, face-to-face contact with significant others,
primarily their colleagues. Clusters of microsystems are called mesosystems (e.g., academics
talking to colleagues from other departments constitutes a linkage between two systems).
Beyond this are settings (i.e., exosystems) that are not experienced directly by the academics,
but nonetheless influence their microsystem through links such as communications from
management. Bronfenbrenner also described a macrosystem, which incorporated the wider
society and culture. The influences here come via policy and reward systems in the
university. Bronfenbrenner further proposed a chronosystem, which captures change over
time in the characteristics of the individual (e.g., career-related transitions) and environmental
change (e.g., national pressure to increase scientific publications and social conditions).
For the individual academic in the research-focused environment, a relational
viewpoint, which focuses on the developing individual in a changing context, is considered a
useful perspective from which to comprehensively understand occupational and career
behaviours (Vondracek, Lerner, & Schulenberg, 1986). These theorists argued that due to the
continually changing nature of the individual and the context, a dynamic interactional
approach, or a developmental contextual perspective, should be applied to understand
occupational and career development. A developmental contextual point of view proposes
that the context is not only continually changing, but also that the changes are influenced by
the individuals and their characteristics. When considered from this perspective, occupation
and career development reflect an interactive process where individuals both affect and are
affected by the features of their environment, including social, cultural, and physical
conditions.
In line with Bronfenbrenner’s (1979, 1986) ecological systems theory and Vondracek
et al.’s (1986) person-context relationships proposition, individuals will function better,
demonstrate adapted outcomes, and be more satisfied when their characteristics fit the
demands of the environment. Individuals with a good person-environment fit are also likely
to receive favourable feedback and input from the surroundings. Conversely, mismatched
individuals will tend to demonstrate poorer outcomes and receive less positive feedback.
Person factors (e.g., personality) and background contextual variables (e.g., socio-
economic status) also shape learning experiences and thus affect occupational and career
behaviours. From a social cognitive perspective (Bandura, 1991; Lent, Brown, & Hackett,
1994), individual interpretations of these experiences shape the development of self-efficacy
(beliefs about one’s ability to successfully manage and perform courses of action) and
outcome expectations (beliefs about the consequences of given actions). Further, self-efficacy
fosters favourable outcome expectations, and both self-efficacy and outcome expectations,
independently and jointly, foster interests (e.g., research interests and activity) and the
development of goals (e.g., intentions to engage in research activities), which, in turn,
motivate research-related actions (e.g., research involvement). The success or failure that
follows these actions promotes further learning, which then prompts individuals to revise
their self-efficacy and outcome expectations, and, in turn, leads to a change in interests and
goals. Thus, providing direct and vicarious research-related experiences, giving the
opportunity to engage in various research tasks, and opening up research possibilities, should
lead to differentiated beliefs about the academic’s own capabilities and consequences of
performing a particular behaviour, which, in turn, should cultivate research-focused interests
and goals that will become more crystallised over time.
2.) On page 7, the authors describe a method for randomly assigning participants to
Samples A and B. They contend that the groups were comparable with respect to several
demographic variables; however, I am most interested in knowing whether the groups were
comparable with respect to institution. Participants came from only four institutions, and it is
likely that participants from the same university would rate their research environments
similarly (given that they are likely experiencing similar pressures). Is it possible that either
Samples A or B consisted of a disproportionately large or small number of faculty from any
one of the four institutions? This might impact the results. Ideally, the sample would have
comprised faculty from a variety of different institutions.
On a related note, it might be interesting to see whether faculty from the same institution had
similar ratings of their research environments. I suppose the ratings might be similar for some
items (availability of funding) but less similar for items that vary more across individuals
(e.g., access to informal mentoring).
Response:
The composition of academics from the four institutions did not differ significantly across
Sample A and Sample B, 2(3) = 6.31, p = .10. We added information regarding this on pages
10 and 17.
With respect to the related note on whether academics from the same university reported
similar ratings of their research environment, we did not calculate this, for while it might be
interesting, we considered it more relevant for future studies, for example, related to whether
different research environments were related to different levels of research output.
3.) The authors note that there were no significant differences between Sample A and
Sample B with respect to age. I’m not sure how meaningful this assertion is, given that
approximately 50% of participants in both samples did not report their ages. This constitutes
a fairly large amount of missing data.
In general, how were missing data handled in this study? This is important to discuss in the
manuscript.
Response:
We retained all cases from participants who completed all of the questionnaire items,
as all participants were academics. Only a small number of responses (24 survey booklets)
had missing scale data, and these were omitted from the analyses.
Some participants did not complete some of their demographic variables. We did not
delete these cases or estimate any of the demographic data, as these data were used primarily
to describe the sample. We added a statement to this effect in the Limitations section:
“We showed that the scale was unrelated to several demographic variables (e.g., age,
gender, tenure, level of education, and institutions), suggesting no inherent bias based on
these characteristics; however, we had missing demographic data, and future studies need to
confirm this, and examine structural invariance on these and other variables to support the
usefulness of the scale.”
4.) I would have liked to have had some more information about the 42 academics included
in the focus groups as well as the four independent reviewers who rated the suitability of the
items (p. 6). For example, who were the reviewers and what qualified them for this task?
Response:
We added these statements on page 6: “The independent reviewers who rated the suitability
of the items consisted of 1 professor in psychology who had expertise in career development
and test development and 3 doctoral-level psychology academics who had expertise in test
development.”
We also added these statements on page 7: “The 42 academics included in the focus groups
consisted of 6 professors, 12 associate professors, 12 assistant professor, 6 lecturers, and 6
junior lecturers…..”
5.) The authors administered a measure of research involvement (i.e., the Research
Involvement Scale). I would be curious to know if Samples A and B differed with respect to
scores on this measure. If one group were more involved in research than the other, the two
samples would not be comparable on a very important dimension (especially given the nature
of the instrument the authors are developing).
Response:
The scores of the Research Involvement Scale for Sample A and Sample B did not differ
significantly, t(596) = -.33 (p = .74). We added this information on page 10.
6.) The authors sampled participants with a wide range of academic roles, including
professors and lecturers. I can imagine that some of these positions are not inherently or
contractually research roles. Would respondents with positions that were non-research
oriented be the best respondents for this type of measure?
Response:
All academics sampled had a research component to their role. The practice of
employing academics who are teaching-only or administration-only is rarely employed in
Indonesia, unlike in some Western countries.
7.) The sample included very few full professors (i.e., approximately 1% of each
subsample), which seems problematic for a couple of reasons. First, how might this have
impacted the authors attempts to determine whether items were responded to differently by
faculty in various positions? Were subsamples for each position (e.g., full professor, associate
professor, assistant professor) large enough (and comparable enough in size) to detect
response differences across groups? Second, full professors may have more institutional
knowledge and research experience than associate and assistant professors. To have so few in
the sample appears to be a notable limitation of this study.
Response:
We added this comment to the Limitation section on page 16:
“We only had a very small number of professors in the samples, and the number of
associate professors, assistant professors, and lecturers were not proportional. Future
studies need to consider the proportion of their sample when collecting data in academics, as
this will affect the response regarding perceived research environment.”
8.) I would have also recommended that the authors collect data regarding the length of
time participants had been at their respective institutions. Newer faculty might not have had
enough time to form opinions about their respective research environments.
Response:
We did have these data, but originally did not consider it as a way to describe the sample. We
have now reported these details in the Participants section on page 9-10.
9.) I would have been interested to see the internal consistency values for the PRES total
score and subscale scores in Sample B. I would recommend the authors report these values.
Response:
We added this ese statements on page 13: “In Sample B, Alpha for the full scale was .93 (M
= 112, SD = 16.65), Factor 1 (α = .90, M = 22.87, SD = 4.29), Factor 2 (α = .94, M =
20.36, SD = 6.22), Factor 3 (α = .87, M = 24.44, SD = 3.70), Factor 4 (α = .91, M = 22.53,
SD = 4.80), and Factor 5 (α = .89, M = 21.79, SD = 4.59).”
10.) For the CFA, the authors suggest that three of the four models (i.e., 2nd order model,
5-factor model, and bifactor model) had satisfactory fit statistics. However, CFI values for the
5-factor and 2nd order models were below .95. Some research has suggested that a more
appropriate criterion for CFI values is .95 or greater (rather than .90 or greater). The authors
may wish to revise their language accordingly. (Please see Hu and Bentler, 1999; reference
provided below.)
Response:
Hair, Black, Babin, and Anderson (2010) provide fit statistics recommendations that are
sensitive to sample size and the number of observed variables to be estimated. For example, a
significant χ2, χ2/df < 3.0, CFI and TLI values > .92, and RMSEA < .08 indicate satisfactory
fit when sample size > 250 and observed variables number between 12 and 30. The CFI
values should be greater than .95 when sample size > 250 and observed variables number
below 12.
Our sample size was 292 and the number of observed variables were 15.
11.) Minor comments:

a. On page 3, the authors write, “Duffy et al. (2013) interview 17 of the most research-
productive counselling psychologists within the American Psychological Association
accredited counselling program.” Did the authors mean 17 faculty across a variety of APA-
accredited programs?
Response:
The sample was comprised of counseling psychology faculty who were the most cumulatively
productive. Duffy et al.’s (2013) study focused on a very specific group of psychologists
within one subfield of psychology, i.e., counseling psychology. We clarified this statement in
the text on page 4.
b. In the implications section, I would further emphasize the potential value of the PRES
for informing organizational change. This point is a good one and should be further
developed. For example, more detail about the specific uses of the instrument for facilitating
organizational improvement would be interesting (and would ultimately make the paper more
compelling).
Response:
We accepted your advice. We added the uses of the instrument for facilitating organizational
improvement in the implication section.
Reference
Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure
analysis. Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1-
55.
References:
Duffy, R. D., Torrey, C. L., Bott, E. M., Allan, B. A., & Schlosser, L. Z. (2013). Time
management, passion, and collaboration: A qualitative study of highly research
productive counseling psychologists. The Counseling Psychologist, 41, 881-917. doi:
10.1177/0011000012457994
Hair, J. F., Black, W. C., Babin, B. J., Anderson, R. E. (2010). Multivariate data analysis. A
global perspective. (7th ed.). New Jersey, NJ: Prentice Hall.
Reviewer: 2

This manuscript reports on the development and psychometric evolution of a measure of
perceived research environment. Specifically, the measure is designed to tap into a construct
comprised of a number of dimensions related to an individual’s perception of the availability
of resources, support, and appreciation of research efforts within the institution in which the
individual is employed. I found the manuscript to be clear and well-written. The authors did
a nice job explaining the concept of the perceived research environment and why it is
important, as well as the previous attempts to at psychometric instruments to capture the
construct. As illustrated in the introduction, previous measures have been quite specific to
either certain fields (e.g., nursing), or populations (e.g., graduate students). Thus, the authors
make a good case for a general perceived research environment scale that could be applicable
to Universities or other research institutions, regardless of the specific discipline. However, I
would recommend the authors add a paragraph at the end of the introduction, before the
“Present Study” section, summarizing this and making this rationale more explicit.
Response:
We accepted your suggestion. We added these statements on page 6: “It is obvious that there
is no general perceived research environment scale suitable for academics. We address this
gap by designing a brief, multidimensional, and psychometrically sound instrument that
could be applicable to universities and other research institutions regardless of specific
disciplines.”
There are other strengths of this paper, including conducting item analyses, evaluation of the
internal structure with EFA followed by a CFA in a random hold-out sample to confirm the
factor structure. The factor analytic methods were appropriate and fit the theoretical
conception of the construct, including use of principal axis factoring, direct oblimin rotation,
which allows for correlations among the rotated factors, and the use of Velicer’s MAP and
parallel analysis to inform the decision on the number of factors to retain. With that said,
there are also some areas that should be addressed to improve the paper and the contribution
of the study.
I appreciate the fact that the authors conducted focus groups as one of the methods for
identifying the important domains of the construct. It would probably be useful if the authors
provided a bit more detail about how the information from the focus groups was analyzed,
and what dimensions they identified, independent of the dimensions that the identified from
the literature review.
Response:
We accepted your suggestion. We added these statements on page 7: “The independent
reviewers who rated the suitability of the items consisted of 1 professor in psychology who
had expertise in career development and test development and 3 doctoral-level psychology
academics who had expertise in test development. They independently reviewed the
discussions and determined the core ideas, and the team met to synthesise the results. The
team identified five salient aspects: of beneficial social relationships, positive reinforcement,
support, encouragement, and role modelling.”
The conduct of the item analyses, including evaluation of item response distributions, item-
total correlations, and inter-item correlations was appropriate, and an often over-looked step
in the process of test development. Although the authors indicated that none of the items
were eliminated based upon the item analyses, it would still be useful information to present
the results of the item analyses in a table, including mean, standard deviation, skewness,
kurtosis, and item-total correlation of each item, as well as the mean item-total correlation, if
not for the original set of items, at least for the final set of items after eliminating items due to
factor loading issues.
Responses:
We accepted your suggestion and added this table:
Table 2
Results of the item analyses
Item Skewness Kurtosis Mean SD Item-total correlation
1. -1.18 2.00 4.57 .93 .62
2. -1.20 2.43 4.61 .97 .64
3. -1.05 1.48 4.56 .97 .51
4. -.73 1.17 4.61 .83 .63
5. -.97 1.16 4.66 .97 .62
6. -.90 .14 4.2 1.35 .63
7. -.81 .10 4.05 1.36 .55
8. -1.17 .63 4.28 1.32 .56
9. -.99 .29 4.21 1.34 .52
10. -.55 -.43 3.88 1.34 .65
11. -.69 .93 4.96 .78 .41
12. -.80 1.17 5.15 .73 .37
13. -.78 .38 4.96 .92 .39
14. -.68 .70 4.90 .83 .50
15. -.58 .79 4.87 .76 .39
16. -1.08 1.32 4.52 1.03 .73
17. -.83 1.10 4.44 1.02 .72
18. -1.08 1.38 4.67 1.08 .65
19. -.81 .71 4.41 1.03 .69
20. -1.08 1.14 4.53 1.08 .74
21. -.34 .03 4.08 1.02 .41
22. -.35 -.46 4.20 1.20 .33
23. -.55 .15 4.35 1.06 .56
24. -.61 .22 4.52 1.01 .58
25. -.69 .91 4.66 .92 .51
For the EFA, the authors indicate that both the minimum average partial (MAP) and parallel
analysis procedures indicated 5 factors, but they do not present the results of the analysis. It
is good practice to provide the numbers generated from these procedures that lead to the
conclusion. So, for the parallel analysis, the first actual and random eigenvalues can be
presented.
Responses:
We added this statement on page 11: “The first eigenvalues are: 14.93, 5.20, 2.38, 1.83, and
1.61.”
Regarding the CFA, the procedures were reasonable, and the fit statistics were
appropriate. The computation of Omega’s was also useful. My one suggestion here would
be to present a table or figure showing the loadings of the bifactor model, which was
determined to be the best fitting model.
Response:
We added these statements on page 14: “Item loadings for factor 1 ranged from .28 to .62,
factor 2 from .80 to .89, factor 3 from .30 to .80, factor 4 from .53 to .80, factor 5 from .32 to
.55. Item loading for perceived research environment ranged from .35 to 77.”
The validity analyses were fairly limited in scope, but the measures used were reasonable and
the correlations found were supportive of construct validity.
The Discussion section was a bit lacking in content. Given that construct validation requires
a multitude of evidence from different methods and perspectives, what is especially needed in
the discussion section is suggestions for further developments, such as other constructs,
measures, and criterion variables would be useful to further establish the nomological
network and construct validity of scores from this measure. I found the statement that the
measure would be useful for “early stages of career counseling” to be perplexing. Perhaps I
am missing something, but it seems this measure would only be applicable to someone who is
already working in an institution with some kind of research focus. Otherwise, what
“research environment” are they reporting on. In addition, it is premature to recommend a
measure for applied use after one development study.
Response:
We accepted your suggestions and revised our statements on page 17:
“Practitioners can use the scale as a diagnostic tool at an early stage of individual’s career
stage as academics, as well as an evaluation instrument in the next career stages.”
And also added these statements on page 17:
“Finally, further developments, such as testing the relationships with other constructs,
measures, and criterion variables would be useful to establish the nomological network and
construct validity of scores from this measure.”
***

Environment Scale
Manuscript ID JPA-18-0139.R1
Fo

Keywords:
rP

ee

Research Environment Scale for use with this population. In Phase 1,
items were developed based on a review of literature, six focus groups,
rR
and expert judgment. In Phase 2, the items were then administered to a

Abstract:
analysis and exploratory factor analysis were used to reduce the number
of items and determine the factor structure. In Phase 3, confirmatory
ev
factor analyses were used on a hold-out sample (N = 292, M age =

43.39) to confirm this structure. In Phase 4, we provided evidence for
construct validity. The practical uses of this newly-developed scale are
discussed.
iew
1
1
2
3
5
6 for Higher Education Academics
7
8 Introduction
9
10
11 Education, research, and services are the three key functions characterizing the
12
13 academic profession in modern-day, higher education systems (Eam, 2015), although
14
16
17
18
around the world, as involvement in research-related activities is recognised as an effective
19
20 means to upgrade a university’s profile (Nguyen, Klopper, & Smith, 2016). Previous studies
21
Fo
23
24
enhances knowledge and competence, and this contributes to high quality research
rP
25
26
27 supervision, which is critical for developing graduate students as independent researchers
28
ee
29 (Lindsay, Breen, & Jenkins, 2002).

30
rR
32
33
34 countries to increase their focus on research, and this tendency has spread to developing
35
ev
36 countries, where research is increasingly viewed as a high priority (Nguyen et al., 2016).
37
iew
39
40
41
research productivity is now a primary consideration in several important organisational
42
44
46
47
research results nationally and internationally in high quality, peer-refereed journals (Nguyen
48
49
50 et al., 2016), researchers have been interested in identifying the predictors of research
51
52 involvement and performance in academics (e.g., Whelan & Markless, 2013).
53
55
56
58
59 led researchers to identify the elements that characterise a good research environment
60
26
1
PERCEIVED RESEARCH ENVIRONMENT SCALE
2
3
4 Table 4
5
Summary Data for Sample B (N = 292; correlations above diagonal)
6
7
Indonesian
8
9 Scale M SD Range α 1 2 3 4 5 6 7 8
10
11 1. Full scale 112.00 16.65 28-145 .93 - .81*** .65*** .63*** .80*** .64*** .47*** .49***
Fo
12
13 2. Subscale 1 (beneficial social relationship) 22.87 4.29 5-30 .90 - .40*** .48*** .52*** .54*** .30*** .35***
r
14
3. Subscale 2 (positive reinforcement) 20.36 6.22 5-30 .94 - .12* .49*** .10* .14* .24**
P
15
e
16 4. Subscale 3 (support and expectations) 24.44 3.70 6-30 .87 - .38*** .49*** .38*** .32***
e
17
18
rR
5. Subscale 4 (focus on research) 22.53 4.80 5-30 .91 - .39*** .54*** .58***
19
20 6. Subscale 5 (positive role models) 21.79 4.59 5-30 .89 - .35*** .27**
e
21
vi
22 7. Organisational culture/ support for research 67.24 15.40 24-144 .93 - .54***
23
e
8. Research involvement 100.82 21.12 18-108 .97 -
24
w
25 Note. * p < .05; ** p < .01; *** p < .001
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
44
45
46
4. Bukti Konfirmasi Review dan
Hasil Review Kedua
(29 Des 2018)
8/1/2020 Gmail - Journal of Psychoeducational Assessment - Decision on Manuscript ID JPA-18-0139.R1

0139.R1
Journal of Psychoeducational Assessment <onbehalfof@manuscriptcentral.com> Sat, Dec 29, 2018 at 12:51 AM
Reply-To: tobin@temple.edu
To:
Cc: dsaklofs@uwo.ca
28-Dec-2018
Dear Dr. :
Thank you for submitting a revision of the Manuscript ID JPA-18-0139.R1 entitled "Development and Initial Validation
of Perceived Research Environment Scale for Higher Education Academics" to Journal of Psychoeducational
Assessment (JPA). I have now received a review from one of the experts in the field who reviewed your initial
submission and has examined your revised manuscript and cover letter. This review is included below for your
reference. I have also carefully read the manuscript and your cover letter in response to reviewers’ comments.
After reviewing these materials, the reviewer and I noted significant improvement over the last version. As you will
see in the review, the reviewer identifies several minor issues. Rather than restating the reviewers’ comments here, I
will simply ask that you carefully read these comments and adjust the manuscript to address them. Once you do so, I
would be pleased to recommend to the Editor, Don Saklofske, that the manuscript be published in JPA.
To revise your manuscript, log into https://mc.manuscriptcentral.com/jopa and enter your Author Center, where you
will find your manuscript title listed under "Manuscripts with Decisions." Under "Actions," click on "Create a
Revision." Your manuscript number has been appended to denote a revision.
You may also click the below link to start the revision process (or continue the process if you have already started
your revision) for your manuscript. If you use the below link you will not be required to login to ScholarOne
Manuscripts.
*** PLEASE NOTE: This is a two-step process. After clicking on the link, you will be directed to a webpage to confirm.
***
https://mc.manuscriptcentral.com/jopa?URL_MASK=15323715e61a4c85abc0a8695a7ea998
You will be unable to make your revisions on the originally submitted version of the manuscript. Instead, revise your
manuscript using a word processing program and save it on your computer.
Once the revised manuscript is prepared, you can upload it and submit it through your Author Center.
When submitting your revised manuscript, you will be able to respond to the comments made by the reviewer(s) in
the space provided. You can use this space to document any changes you make to the original manuscript. In order
to expedite the processing of the revised manuscript, please be as specific as possible in your response to the
reviewer(s).
IMPORTANT: Your original files are available to you when you upload your revised manuscript. Please delete any
redundant files before completing the submission.
Because we are trying to facilitate timely publication of manuscripts submitted to Journal of Psychoeducational
Assessment, your revised manuscript should be submitted within 30 days from receipt of this letter. If it is not
possible for you to submit your revision in this amount of time, please advise the Associate Editor before the 30 day
period as the link to you article will expire and you will not be able to re-submit your paper without making a specific
request.
Once again, thank you for submitting your manuscript to Journal of Psychoeducational Assessment and I look
forward to receiving your revision.
Sincerely,
Dr. Renée Tobin

tobin@temple.edu
Don Saklofske
Editor
Reviewer: 1

Thank you for the opportunity to review this revised manuscript (JPA-18-0139.R1). Overall, I appreciate the authors’
responsiveness to reviewer feedback and believe their edits have much improved the manuscript. I hope the authors
will find the following comments helpful as they continue to revise their manuscript.
1. I greatly appreciated the authors’ expansion of their theoretical rationale in the introduction. However, I would
recommend removing the paragraph on the social cognitive perspective (p. 3-4). The study is already situated in the
work of Bronfenbrenner (1979) and Vondracek et al. (1986), and applying too many theoretical perspectives makes it
hard to distill the study’s rationale. Moreover, the social cognitive perspective’s emphasis on self-efficacy and other
specific individual variables seems less relevant to this study, given that the PRE is intended to measure
characteristics of the research environment (and not of the individual academic).
2. In their response to reviewers, the authors noted that all academics in this setting had research components to
their roles. I would recommend stating this in the manuscript.
3. I appreciated your analysis of potential differences in institutional affiliation between Samples A and B. I also
appreciated your reporting internal consistency values in Sample B.
4. On page 16, the authors state that practitioners can use the PRE as a diagnostic tool for individuals. I am
unclear as to why and how this would be useful. Many academics have little control over their research environments.
It seems to me that this scale would be much more useful for conducting systems-level needs assessments and
planning for departmental/organizational change. I would emphasize these potential applications (rather than
emphasizing potential applications for individual academics).
5. Minor comments
a. While the manuscript is generally well-written, its first two sentences are “run-on” sentences. I would
recommend revising them.
b. On page 6 (line 43), the sentence beginning with “it is obvious that” could be revised as follows: “To the authors’
knowledge, a perceived research environment scale suitable for academics has yet to be published in the peer-
reviewed literature.”
c. When referring to internal consistency values, please write “Cronbach’s alpha” rather than just “alpha.”
d. On page 4 (line 52), the phrase “or at hobbies” should read “or engaged in hobbies.”
e. In describing the OCSR and the RI measures (p. 10), please use complete sentences (e.g., “A sample item
from this measure is…”)
5. Bukti Konfirmasi Submit Revisi Kedua,
Respon kepada Reviewer,
dan Artikel yang Diresubmit
(2 Jan 2019)
8/1/2020 Gmail - Journal of Psychoeducational Assessment - Manuscript ID JPA-18-0139.R2
Journal of Psychoeducational Assessment - Manuscript ID JPA-18-0139.R2

Journal of Psychoeducational Assessment <onbehalfof@manuscriptcentral.com> Wed, Jan 2, 2019 at 3:24 PM
To:
02-Jan-2019
Dear Dr. :
Your manuscript entitled "Development and Initial Validation of Perceived Research Environment Scale for Higher
Education Academics" has been successfully submitted online and is presently being given full consideration for
publication in Journal of Psychoeducational Assessment.
Your manuscript ID is JPA-18-0139.R2.
Sincerely,
RESPONSE TO THE REVIEWER
OF THE JOURNAL OF PSYCHOEDUCATIONAL ASSESSMENT
MANUSCRIPT ID: JPA – 18 – 0139. R1
Reviewer: 1

Thank you for the opportunity to review this revised manuscript (JPA-18-0139.R1). Overall,
I appreciate the authors’ responsiveness to reviewer feedback and believe their edits have
much improved the manuscript. I hope the authors will find the following comments helpful
as they continue to revise their manuscript.
1. I greatly appreciated the authors’ expansion of their theoretical rationale in the

introduction. However, I would recommend removing the paragraph on the social cognitive
perspective (p. 3-4). The study is already situated in the work of Bronfenbrenner (1979) and
Vondracek et al. (1986), and applying too many theoretical perspectives makes it hard to
distill the study’s rationale. Moreover, the social cognitive perspective’s emphasis on self-
efficacy and other specific individual variables seems less relevant to this study, given that
the PRE is intended to measure characteristics of the research environment (and not of the
individual academic).
Response:
We accepted your suggestion. We removed the paragraph on the social cognitive perspective
(p. 3-4)
2. In their response to reviewers, the authors noted that all academics in this setting had
research components to their roles. I would recommend stating this in the manuscript.
Response:
We accepted your suggestion. We stated it in the Participants section on page 9.
3. I appreciated your analysis of potential differences in institutional affiliation between

Samples A and B. I also appreciated your reporting internal consistency values in Sample B.
Response:
Thank you for your previous suggestion.
4. On page 16, the authors state that practitioners can use the PRE as a diagnostic tool for
individuals. I am unclear as to why and how this would be useful. Many academics have little
control over their research environments. It seems to me that this scale would be much more
useful for conducting systems-level needs assessments and planning for
departmental/organizational change. I would emphasize these potential applications (rather
than emphasizing potential applications for individual academics).
Response:
We accepted your suggestion. On page 16, we revised our statement: “Practitioners can use
the scale for conducting systems-level needs assessments and planning for
departmental/organisational change.”
5. Minor comments
a. While the manuscript is generally well-written, its first two sentences are “run-on”
sentences. I would recommend revising them.
b. On page 6 (line 43), the sentence beginning with “it is obvious that” could be revised as
follows: “To the authors’ knowledge, a perceived research environment scale suitable for
academics has yet to be published in the peer-reviewed literature.”
c. When referring to internal consistency values, please write “Cronbach’s alpha” rather
than just “alpha.”
d. On page 4 (line 52), the phrase “or at hobbies” should read “or engaged in hobbies.”
e. In describing the OCSR and the RI measures (p. 10), please use complete sentences
(e.g., “A sample item from this measure is…”)
Response:
We accepted your suggestions.
a. We revised the first two sentences: “Education, research, and services are the three key
functions characterising the academic profession in modern-day, higher education
systems (Eam, 2015). However, academic research and publications have been
increasingly emphasised at most universities around the world, as involvement in
research-related activities is recognised as an effective means to upgrade a university’s
profile (Nguyen, Klopper, & Smith, 2016).”
b. We revised the sentence on page 6 (line 43): “To the authors’ knowledge, a perceived
research environment scale suitable for academics has yet to be published in the peer-
reviewed literature.”
c. We revised “alpha” to “Cronbach’s alpha” when referring to internal consistency
values on page 10, 12, and 13.
d. On page 4 (line 52), we revised “or at hobbies” to “or engaged in hobbies.”
e. In describing the OCSR and the RI measures on page 10, we revised our previous
sentences to complete sentences: “A sample item from this measure is…”
***

Environment Scale for Higher Education Academics
Manuscript ID JPA-18-0139.R2

Fo

Keywords:
rP

ee
Research Environment Scale for use with this population. In Phase 1,

items were developed based on a review of literature, six focus groups,
and expert judgment. In Phase 2, the items were then administered to a
Abstract:
rR
analysis and exploratory factor analysis were used to reduce the number
of items and determine the factor structure. In Phase 3, confirmatory
factor analyses were used on a hold-out sample (N = 292, M age =
43.39) to confirm this structure. In Phase 4, we provided evidence for
ev
construct validity. The practical uses of this newly-developed scale are

discussed.
iew
1
1
2
3
5
6 for Higher Education Academics
7
8 Introduction
9
10
11 Education, research, and service are the three key functions characterising the
12
13 academic profession in modern-day, higher education systems (Eam, 2015). However,
14
16
17
18
around the world, as involvement in research-related activities is recognised as an effective
19
20 means to upgrade a university’s profile (Nguyen, Klopper, & Smith, 2016). Previous studies
21
Fo
23
24
enhances knowledge and competence, and this contributes to high quality research
rP
25
26
27 supervision, which is critical for developing graduate students as independent researchers
28
ee
29 (Lindsay, Breen, & Jenkins, 2002).

30
rR
32
33
34 countries to increase their focus on research, and this tendency has spread to developing
35
ev
36 countries, where research is increasingly viewed as a high priority (Nguyen et al., 2016).
37
iew
39
40
41
research productivity is now a primary consideration in several important organisational
42
44
46
47
research results nationally and internationally in high quality, peer-refereed journals (Nguyen
48
49
50 et al., 2016), researchers and administrators have been interested in identifying the predictors
51
52 of research involvement and performance in academics (e.g., Whelan & Markless, 2013).
53
55
56
58
59 led researchers to identify the elements that characterise a good research environment
60
26
1
PERCEIVED RESEARCH ENVIRONMENT SCALE
2
3
4 Table 4
5
Summary Data for Sample B (N = 292; correlations above diagonal)
6
7
Indonesian
8
9 Scale M SD Range α 1 2 3 4 5 6 7 8
10
11 1. Full scale 112.00 16.65 28-145 .93 - .81*** .65*** .63*** .80*** .64*** .47*** .49***
Fo
12
13 2. Subscale 1 (beneficial social relationship) 22.87 4.29 5-30 .90 - .40*** .48*** .52*** .54*** .30*** .35***
r
14
3. Subscale 2 (positive reinforcement) 20.36 6.22 5-30 .94 - .12* .49*** .10* .14* .24**
P
15
e
16 4. Subscale 3 (support and expectations) 24.44 3.70 6-30 .87 - .38*** .49*** .38*** .32***
e
17
18
rR
5. Subscale 4 (focus on research) 22.53 4.80 5-30 .91 - .39*** .54*** .58***
19
20 6. Subscale 5 (positive role models) 21.79 4.59 5-30 .89 - .35*** .27**
e
21
vi
22 7. Organisational culture/ support for research 67.24 15.40 24-144 .93 - .54***
23
e
8. Research involvement 100.82 21.12 18-108 .97 -
24
w
25 Note. * p < .05; ** p < .01; *** p < .001
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
44
45
46
6. Bukti Konfirmasi Artikel Accepted
(16 Jan 2019)

0139.R2
Journal of Psychoeducational Assessment <onbehalfof@manuscriptcentral.com> Wed, Jan 16, 2019 at 3:28 AM
Reply-To: tobin@temple.edu
To:
Cc:
15-Jan-2019
Dear Dr. :
I have now carefully read the revised manuscript and your cover letter in response to reviewer comments. After
reviewing these materials, I noted that you addressed all issues raised in the last round of reviews. Thus, it is a
pleasure to accept your manuscript entitled "Development and Initial Validation of Perceived Research Environment
Scale for Higher Education Academics" in its current form for publication in Journal of Psychoeducational
Assessment.
Thank you for your fine contribution. On behalf of the Editors of Journal of Psychoeducational Assessment, we look
forward to your continued contributions to the Journal.
Sincerely,
Dr. Renée Tobin
tobin@temple.edu
Don Saklofske
Editor
7. Bukti Konfirmasi Artikel Published Online
(11 Feb 2019)
8/1/2020 Gmail - Your article is now published online
Your article is now published online

SAGE Journals <noreply@sagepub.com> Mon, Feb 11, 2019 at 1:50 PM
Reply-To: noreply@sagepub.com
To:
Cc: sage.eprints@sagepub.com
Dear Diponegoro University,
Thank you for publishing your article with SAGE Publishing and Journal of Psychoeducational Assessment. Your
article “Development and Initial Validation of Perceived Research Environment Scale for Higher Education Academics
is now published and your complimentary e-copy is available at https://journals.sagepub.com/
eprint/XAezNJfPJRHnjn49D3Bc/full.
The above link and final published PDF are only for your personal use and non-profit teaching purposes and should
not be posted online or otherwise distributed. If you wish to post your article on your personal website or institutional
repository, you may use the accepted version. Please see the SAGE author sharing guidelines, linked below, for full
details about appropriate use and version definitions.
The permanent link for your article is "https://doi.org/10.1177/0734282919828892".
Please also visit our online resources for more information:
SAGE author sharing guidelines,

Promoting your article, including maximizing its impact via Kudos,
Post-publication corrections,
Author gateway.
Thank you again for publishing with SAGE and Journal of Psychoeducational Assessment. We value your feedback
and encourage you to complete the brief Journal Author Survey you will receive shortly.
Best wishes,
SAGE Journals Author Services
authorproductionqueries@sagepub.co.uk
Note: Please do not reply to this message, as replies are routed to an unmonitored mailbox. If you experience
difficulty with the provided links, please check that the full link transferred to your browser without any line breaks or
errant characters such as < or >. You may receive this message twice, once for your article’s initial OnlineFirst
publication and a second time for your article’s issue publication. Please direct any questions to
authorproductionqueries@sagepub.com, noting the article title and journal title.

Korespondensi SosHum

Uploaded by

Copyright:

Available Formats

You might also like

Korespondensi SosHum

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Korespondensi SosHum

Uploaded by

Copyright:

Available Formats

BUKTI KORESPONDENSI

ARTIKEL JURNAL INTERNASIONAL BEREPUTASI

Judul artikel : Development and Initial Validation of Perceived Research

No. Perihal Tanggal

1. Bukti konfirmasi submit artikel dan artikel yang 26 Juni 2018

Journal of Psychoeducational Assessment - Manuscript ID JPA-18-0139

Your manuscript ID is JPA-18-0139.

Thank you for submitting your manuscript to Journal of Psychoeducational Assessment.

Development and Initial Validation of Perceived Research

Journal: Journal of Psychoeducational Assessment

Manuscript Type: Regular Article

perceived research environment, scale development, academics,

There is a growing interest in the perceived research environment for

higher education academics, we designed and validated the Perceived

sample of Indonesian academics (N = 306, M age = 42.29 years). Item

analyses were used on a hold-out sample (N = 292, M age = 43.39) to

validity. The practical uses of this newly-developed scale are discussed.

25 which is critical for developing graduate students as independent researchers (Lindsay,

27 We employed a standard, classic test development approach (cf. DeVellis, 2016) to

27 academics: beneficial social relationships, positive reinforcement, support and expectations,

27 (positive role models); 6-point-scale of 1 = strongly disagree to 6 = strongly agree.

27 professional psychology. Professional Psychology: Research and Practice, 24, 468-

27 with research productivity of agricultural education faculty. Journal of Agricultural

27 Factors influencing research productivity of business faculty. International Journal of

Journal of Psychoeducational Assessment - Decision on Manuscript ID JPA-18-

Reviewer(s)' Comments to Author:

Comments to the Author

11.) Minor comments:

Comments to the Author

Journal of Psychoeducational Assessment - Manuscript ID JPA-18-0139.R1

Your manuscript ID is JPA-18-0139.R1.

Thank you for submitting your manuscript to Journal of Psychoeducational Assessment.

MANUSCRIPT ID: JPA – 18 – 0139

Comments to the Author

11.) Minor comments:

Comments to the Author

Development and Initial Validation of Perceived Research

Journal: Journal of Psychoeducational Assessment

Manuscript Type: Regular Article

perceived research environment, scale development, academics,

There is a growing interest in the perceived research environment for

higher education academics, we designed and validated the Perceived

and expert judgment. In Phase 2, the items were then administered to a

factor analyses were used on a hold-out sample (N = 292, M age =

29 (Lindsay, Breen, & Jenkins, 2002).

Journal of Psychoeducational Assessment - Decision on Manuscript ID JPA-18-

Associate Editor, Journal of Psychoeducational Assessment

Reviewer(s)' Comments to Author:

Comments to the Author

Journal of Psychoeducational Assessment - Manuscript ID JPA-18-0139.R2

Your manuscript ID is JPA-18-0139.R2.

Thank you for submitting your manuscript to Journal of Psychoeducational Assessment.

MANUSCRIPT ID: JPA – 18 – 0139. R1

Comments to the Author

1. I greatly appreciated the authors’ expansion of their theoretical rationale in the

3. I appreciated your analysis of potential differences in institutional affiliation between

Development and Initial Validation of Perceived Research