Development and Validation of A Questionnaire For Measuring Perceived Political

J O U R N A L OF ORGANIZATIONAL BEHAVIOR, VOL.
17, 179-190 (1996)
Development and validation of a

questionnaire for measuring
perceived political considerations
in performance appraisal
AHARON TZINER
Utiiwrsity of' Mo/itriul
GARY P. LATHAM
Univrrsit,v of' Toronto
BRUCE S. PRICE
Concordki Univrrsitv
AND
ROBERT HACCOUN
Univcwiry Uf' Motitrid
Summary A 25-item scoring instrument was developed to measure perceptions of the extent to
which performance appraisals are affected by organizational politics. The survey was
administered to 157 francophone managers. A factor analysis revealed that the
perceptions of the extent to which political considerations affect the appraisal an
employee receives can be sensibly interpreted in terms of a single general factor. The
internal consistency and test-retest reliability of the instrument was high. Moderate
support was found for the instrument's convergent and discriminant validity.
Introduction
A recent survey of raters, ratees and administrators of performance appraisal systems revealed
that the majority of respondents from each of these groups feels that deliberate distortion of
performance rating by raters has more to d o with rating inaccuracy than inadvertent, cognitive
errors committed by raters (Bernardin and Villanova, 1986). This same research also indicated
that the vast majority of empirical studies have concentrated on non-deliberate rating distortion.
Researchers have concentrated, for example, on rater training programs which assume that
raters would rate accurately if only they had the skill to d o so (e.g. Pulakos, 1986). Frame of
Addressee for correspondence: Aharon Tziner, Department of Psychology. Universitk de Montreal, Montreal (Quebec).
Canada H3C 357.
CCC 0894-3796/96/020 I 79 -1 2 Rewivrtl IS Mnrcli lYY4

C1 1996 by John Wiley & Sons, Ltd. Accepted 14 Mrirch 1995
10991379, 1996, 2, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/(SICI)1099-1379(199603)17:2<179::AID-JOB740>3.0.CO;2-Z by Egyptian National Sti. Network (Enstinet), Wiley Online Library on [28/03/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
180 A . TZINER ET A L .
reference training, diary-keeping, and rater error training all assume that raters will rate more
accurately with these cognitive aids. There is some evidence that this is not the case. Recently
Prince and Mohrman ( 1989) found that, although interpersonal closeness appears to reduce the
actor-observer cognitive bias, being interpersonally close appears to reduce rating agreement in
manager-subordinate dyads. They conclude that non-cognitive situational factors appear to
overwhelm any gain associated with reduced cognitive bias. Lack of inter-rater agreement on
many aspects of performance appraisal seems to be the norm (Prince, Lawler and Mohrman,
1991; Prince, 1991 ). This general pattern has led Wexley and Klimoski (1984) to conclude that it
is unrealistic to expect high agreement (and, by inference, high validity) on the performance
rating. However, in general, there has been little attempt to study the worker characteristics
related to deliberate rating distortion.
Bernardin and Beatty ( 1984) have proposed that rater perceptions of organizational politics
may be related to rater accuracy and to the acceptance of the performance appraisal system.
Organizational politics is an integral aspect of organizational life and relates to power, authority
and influence. Power is defined as an attempt to influence others (Cobb, 1984; Pfeffer, 1981) and
the ability to mobilize resources. energy and information on behalf of a preferred goal or strategy
(Tushman. 1977). The intent of organizational politics is (a) to protect and/or enhance an
individual's self-interests; or (b) to further another person's or a group's interests or goals,
through legitimate as well as non-sanctioned means (Altman, Valenzi and Hodgetts, 1985).
Political behavior and the use of power affect almost every important decision in organizations
(Pfeffer, 1981).
In summary. this research argues that performance appraisal does not take place in a vacuum:
rather, it occurs in the context of rater's pursuit of specific goals. Such goals include, according to
Cleveland and Murphy (1992): ( I ) projecting a favorable image of their units in terms of
effectiveness and efficiency that will. of course, reflect on them as successful managers; (2)
procuring for themselves valuable organizational 'goodies' such as access to central networks of
information and decision making, prestigious assignments and appointments, bonuses, and so
on; (3) portraying an image of a caring boss; (4) avoiding negative consequences and confron-
tations with employees; and (5) avoiding disapproval from peers. It is suggested that the raters
first establish these personal goals and then intentionally distort the performance ratings (which
are then, of course, inaccurate), in order to materialize these goals. Examples of these distortions
are inflation of performance ratings of subordinates and avoidance of scoring low performance
ratings.
Mayes and Allen (1977) would conceive of such acts as manifestations of what they call
Orgariirutiorial Politicx, which they define as 'the management of influence to obtain ends not
sanctioned by the organization or to obtain sanctioned ends through non-sanctioned means'
(p. 657).
Although the wider use of organizational politics has been reviewed in the literature (Pfeffer.
1981). generally speaking, the subject has barely been studied in the context of performance
appraisal. However. Longenecker, Sims and Gioia (1987) and Longenecker and Gioia (1988)
found deliberate manipulation of ratings for political purposes in their investigations of
performance appraisal conducted by managers: specifically, such errant behavior was found to
have been used by managers to obtain rewards for subordinates (e.g. maximized merit increases)
or to acquire additional resources for their units.
Moreover. in Longenecker rt d . ' s ( 1987) investigation, executives reported believing that, in
certain circumstances, it would be both in their best interest arid justifiable to manufacture
inaccurate performance ratings in order to maintain a positive work group climate or to avoid
negative outcomes for their ratees (e.g. demotions, transfers to other units, lay-offs, etc.).
POLITICAL C O N S I D E R A T I O N S I N P E R F O R M A N C E A P P R A I S A L 181
Additionally, the supervisors felt that rating accuracy was ‘not important’ since in their view the
appraisal process serves as no more than a discretionary tool for motivating and rewarding
subordinates.
Bjerke. Cleveland, Morrison and Wilson ( 1987) achieved similar results from interviews with
Navy officers (raters) as well as from a small sample of industry personnel managers. Further-
more, Mohrman and Lawler (1983) and Prince et a/. (1991) have also pointed out that when
rating the performance of subordinates, managers are most likely to use discretion regarding
those with whom they have had a close working relationship or social ties.
These latter investigations have indeed attempted to tackle the theme of how politics relates to
performance appraisal. However, it is clear that the treatment of this issue has not been pursued
within a systematic framework or quantitative methodology, since most of the empirical data in
these studies has emanated from open-ended interviews conducted with executives.
In order to remove this drawback, a structured instrument must be developed capable of
quantitatively measuring the perceptions of the extent to which particular political consider-
ations are manifest in performance appraisal. Therefore, the objective of this study is to develop
and assess the psychometric qualities of such an instrument.
Instrument development
The instrument devised to address the research agenda presented above is called the Question-
naire of Political Considerations in Performance Appraisal (QPCPA). We generated a pool of
items capitalizing on: (a) our perusal of conceptualizations, statements and suggestions that the
literature has to offer on various political considerations likely to operate on the rater while
formulating performance evaluations (e.g. Longenecker ef a/., 1987; Murphy and Cleveland,
1991); (b) empirical research on motivations to dissemble performance ratings (Banks and
Murphy, 1985; Gioia and Sims, 1985; Hogan, 1987; Kane and Kane, 1992); (c) observations
emanating from our organizational consulting experience of factors contributing to intentional
biasing of ratings; and (d) the general literature on organizational power, politics and decision
making processes (e.g. Daft, 1986; Bedeian and Zammuto, 1991). Thirty (30) items were
generated for the initial pool with respect to which the authors of the present article have reached
consensus as to clarity, appropriateness and content validity. Items incorporate such considera-
tions as a rater’s attempts to enhance her/his organizational posture, to avoid conflicts with
employees, to conform with social pressures, to settle personal disputes, to attain personal
benefits and so forth. The full list of items is presented in Table 1.
Phase 1: preliminary analysis

Pretesting
Following the back-translation procedure an equivalent French version of the 30-item
instrument was designed. It was then administered to a randomly-selected sample of employees
( N = 51) occupying managerial positions at a French-speaking institution located in Montrtal.
The instrument required the subjects to indicate the extent to which the content of each of the
30 items is, in their perception, descriptive of raters’ considerations manifest in appraising
performance of their employees in the current organization. Responses were recorded on a six-
point Likert-type scale ranging from ‘I’ (very atypical) to ‘6’ (very typical). Simultaneously the
subjects were also administered the French version of the Social Desirability Scale (Wright,
182 A . T Z I N E R ET AL.
Richer, Lecours and Boucher, 1987) in order to explore whether subjects' responses to QPCPA's
items were affected by social desirability considerations.
Item selection
The objective at this phase of the device construction was to pinpoint those items which
represented in the most accurate and appropriate manner the area of study; that is, political
considerations perceived as prevailing in the process of performance ratings formulation.
Two criteria were used to decide which items would be retained: (a) an item-total correlation
of 0.55, at least; and (b) a lack of significant correlation with the Social Desirability Scale (Crown
and Marlowe's Social Desirability Scale, Strahan and Gerbasi, 1972).
In doing so. we followed Kumar and Beyerlein's (1991) example, although, for the sake of
rigor, we set a stronger item-total correlation than they did. The screening process based on
these criteria resulted in the elimination of five items, leaving the instrument with 25 items. Their
item-total correlations ranged from 0.55 to 0.81 and their correlations with the Social
Desirability Scale (alpha = 0.79) varied between 0.01 and 0.20 (none being statistically
significant, that is p > 0.05). The means, standard deviations, item-total correlations and the
correlations with the total score on the Social Desirability Scale for the 30-item QPCPA are
displayed in Table 1 .
Phase 2: structure, reliability and validity analyses

In order to explore the psychometric properties of the 25-item QPCPA instrument, it was
administered to all the employees in managerial positions in the same organization who had not
participated at Phase 1. They were requested to simultaneously respond to four other question-
naires---the French versions of Need for power, the Machiavellianism, State-Trait Anxiety and
Organizational commitment inventories1-in order to permit evaluation of convergent and
discriminant validities of the QPCPA.
Of all the subjects whose participation had been solicited 157 (i.e. 60 per cent) completed and
returned their questionnaires. The sequence of the items from each questionnaire was alternated
so that possible order effect was precluded. Fifty-six per cent of the respondents were men and 44
per cent were women. All were employed full-time, their average age was 46.6 years (S.D. = 7.3
years), 56 per cent held an academic degree and 28 per cent had completed partial academic
studies.
Their experience as raters was on average 10 years (S.D. = 8.89 years) and their seniority in
managerial positions was on average 13.09 years (S.D. = 8.84 years). They had, on average, 15
subordinates reporting to them.
Factor analysis
To determine whether the structure of the QPCPA was homogeneousjunidimensional or
multidimensional, a principal component analysis rotated to a varimax solution was performed.
The factor analysis yielded a three-factor solution, accounting for 69.4 per cent of the variance.
Table 2 describes the results of this analysis. While following the example of other researchers
(for instance, Campion, Cheraskin and Stevens, 1994; Morrison, 1994), in setting a loading of
' The English version of the questionnaire was translated into French using the back-translation method.
POLITICAL CONSIDERATIONS IN PERFORMANCE APPRAISAL 183
Table I . Means, standard deviations. item-total correlations and correlation with the social desirability
score for the Questionnaire of Political Considerations in Performance Appraisal (QPCPA)
I tem M S.D. Item-total Correlation with
r social desirability
1. Supervisors avoid giving performance 2.84 1.39 0.71 0.02
ratings that may antagonize employees (3.25) ( 1.43) (0.51)
(e.g. a low rating) [PCI]
2. Supervisors are giving low performance 2.90 I .49 0.66 0.01
ratings because they fear that their (2.90) (1 .38) (0.67)
employees will try to be transferred to
another boss [PC2]
3. Supervisors avoid giving a low performance 2.00 1.13 0.49 0.03
rating because they fear violent behavior
on the part of their employees'"'
4. Supervisors inflate performance ratings 2.55 1.47 0.64 0.04
of those employees who are able to (2.67) (1.43) (0.73)
procur them special services, favors, or
benefits [PC3]
of employees who have access to valuable (3.35) ( I .45) (0.74)
sources of information [PC4]
6. Supervisors' performance ratings reflect 2.20 1.30 0.77 0.20
in part their personal liking or disliking (2.68) (1.33) (0.78)
of the employees [PC5]
7. Supervisors' performance ratings are 2.08 1.16 0.73 0.20
affected by the extent to which employees (2.52) (1.33) (0.78)
are perceived as sharing the same basic
values as they do [PC6]
8. The performance ratings of employees 3.29 1.66 0.56 0.1 1
are affected by their ability to inspire their (3.30) (1.54) (0.64)
enthusiasm to the supervisor who rates
their performance [PC7]
9. Supervisors give performance ratings 2.76 1.56 0.63 0.003
that will make them look good to their (2.78) (1.36) (0.52)
superiors [PCS]
10. The quality of the supervisor-employee 3.25 1.49 0.69 0.09
personal relationship throughout the (3.17) (1.40) (0.69)
rating period (e.g. tense-relaxed, trusting-
distrusting, friendly-hostile) affects the
performance rating [PC9]
11. Supervisors are likely to give an inflated 3.83 I .39 0.68 0.04
performance rating in order to avoid (3.28) (1.44) (0.67)
negative/uncomfortable feedback sessions
with their employees [PCIO]
12. Supervisors avoid giving performance 3.22 1.66 0.61 0.08
ratings which may have negative (3.03) (1.43) (0.74)
consequences for the employee
(e.g. demotion, lay-off, no bonus,
salary freeze, etc.) [PCI I]
13. Supervisors inflate performance ratings in 2.20 1.20 0.74 0.08
order to maximize rewards offered to their (2.39) (1.21) (0.70)
employees (e.g. salary increases promotions,
prestigious assignments, etc.) [PC12]
14. Supervisors inflate performance ratings I .73 0.80 0.50 0.04
in order to maintain a positive image of
their department or organization'"' to others Table continued on rie.ut page
184 A. TZINER ET A L
Table 1 (continued)
Item M S.D. Item-total Correlation with
15. Supervisors produce accurate 2.47 1.36 0.72 0.07
performance ratings only to the extent (2.75) ( I .38) (0.84)
that they perceive that they may be
rewarded for doing so or penalized for
failing to do so [PC 131
16. Supervisors produce accurate 2.04 1.22 0.80 0.06
performance ratings only to the extent (2.42) (1.36) (0.78)
that they perceive that this is the norm
in the organization [PC14]
17. Employees holding a high status- 2.43 1.33 0.78 0.07
position in their organization will get a (2.72) (1.43) (0.83)
higher performance rating than is deserved
(i.e. regardless of their real performance,
employees' ratings are affected by their
status in their organization) [PC15]
18. Supervisors give high performance 3.06 1.33 0.61 0.0 I
ratings because they believe that their (2.90) (1.35) (0.83)
employees have already passed through
many organizational hurdles and
therefore are highly competent [PCI6]
19. In assigning ratings, supervisors conform 2.39 1.27 0.8 1 0.07
to what they believe is the norm (2.52) ( I .33) (0.83)
('acceptable') in their organization so as
to avoid disapproval by their peers [PC17]
20. Supervisors give low performance ratings 2.76 1.54 0.80 0.02
to teach rebellious employee a lesson (3.16) (1.47) (0.78)
[PC I X]
71. Supervisors give low performance ratings 3.06 1.49 0.20 0.04
to encourage an employee to leave the
organization'"'
22. Supervisors use performance ratings 2.69 1.50 0.68 0.22
to send a message to their employees (2.50) (1.30) (0.62)
(e.g. encourage risk-taking, creativity,
etc.) [PC19]
23. Supervisors give accurate performance 4.04 1.44 0.52 0.05
ratings only to the extent that they think
they will be rewarded for doing so")
of those employees who possess (2.71) (1.33) (0.85)
special characteristics (e.g. high
popularity, compliancy, etc.) [PC20]
25. The fear that performance ratings 2.41 1.10 0.69 0.04
may threaten the self-esteem of employees (2.72) (1.34) (0.79)
discourage supervisors from giving
negative-though accurate-ratings [PC2 I]
26. Supervisors give higher performance 2.50 1.25 0.78 0.04
ratings than is deserved in order to (2.76) ( I .40) (0.84)
gain supportkooperation from their
employees (PC221
ratings than is deserved in order to (2.63) ( I .40) (0.73)
repay favors to their employees [PC23]
Tnhlr continued oti next page
Table I (continued)
I tem M S.D. Item-total Correlation with
____
28. Supervisors give equivalent performance 2.59 I .70 0.72 0.08

ratings to all their employees in order to (2.68) (1.32) (0.83)
avoid resentment and rivalries among
them [PC24]
ratings than is deserved to those employees (2.86) ( I .42) (0.75)
who control valuable organizational
resources [PC25]
30. Supervisors’ performance ratings are 2.06 1.25 0.54 0.13
determined in part by the power of the
employees (i.e. control of decisions,
expertise in certain areas, indispensability,
centrality in the information network, etc.)‘”’
Total 78.90 26.33
(70.61) (26.51)
~~ ~~~ ~ ~~ ~~ ~~
The figures in brackets derive from the Phase 2 respondents ( N = 157) whereas the remainder came from the Phase I
subjects ( N = 51). Responses to each of the items ranged from ‘I’ (very atypical) to ‘6’ (very typical) of the reality in the
respondents’ organization.
The items marked ‘(a)’ were those dropped after Phase 1.
0.50 and above in order to assign an item to a factor, we realize, however, that: (a) a large number
of items loaded very high on the first factor (accounting for roughly 60 per cent of the variance):
this factor may be labeled ‘Instrumentality (manipulations to acquire benefits)’ consideration;
(b) the second factor which may be labelled ‘Attaining and exercising control’ considerations,
accounts for an additional 5.4 per cent of variance; and (c) the third factor, labeled ‘Social-
interpersonal’ considerations, has emerged as being relatively marginal, explaining only 4.4 per
cent of the item variance. We can thus conclude that the structure of the QPCPA can be sensibly
interpreted in terms of a single general factor. This conclusion evidently gains further support
since the internal consistency (Cronbach’s alpha) of the entire scale was 0.97. Even though this
may partly be due to the number of items, it suggests that the items represent the same construct
domain. Moreover, the eigenvalues make a dramatic scree plot that also argues for a single
general factor2. This empirical finding confirms, in fact, in the context of performance appraisal,
the coherent structure of the general concept of organizational politics (Mayes and Allen, 1977;
Pfeffer, 1981).
Reliability
Two approaches were employed to establish the reliability for the instrument: (a) internal
consistency; and (b) stability over time (an interval of two weeks between the two administra-
t i o n ~ ) ~It . is worth noting that the internal consistency for the second administration was
The authors are indebted to Daniel C. Ganster for suggesting this additional reasoning.
In order to ensure anonymity and yet make it possible to match subjects responses to the first and the second
administration of the QPCPA, each respondent was requested to mark on the questionnaire a personalized identity code
composed of the first three digits of his/her social security number and the last four digits of his/her residence telephone
number.
186 A. TZINER ET A L
Table 2. Factor analysis of the 25-item Questionnaire of Political Considerations in Performance Appraisal
Item Communality Factor loadings
Factor I Factor I1 Factor I l l
(manipulation to Attaining & exercising Social/
acquire benefits) control interpersonal
- ~
PC 1 0.68 0.60 0.55 0.09

PC2 0.65 0.6Y 0.39 0.12
PC3 0.74 0.75 0. I4 0.38
PC4 0.65 0.76 0.06 0.26
PC5 0.73 0.80 0.03 0.31
PC6 0.77 0.82 0.18 0.24
PC7 0.45 0.66 0.0 1 0.09
PCX 0.63 0.55 0.58 0.06
PCY 0.60 0.71 0.29 0.11
PClO 0.57 0.69 0.29 0.07
PCI 1 0.66 0.76 0.24 0.12
PC12 0.74 0.81 0.20 0.20
PC13 0.77 0.86 0.16 0.09
PC14 0.65 0.80 0.03 0.1 1
PC15 0.74 0.85 0.06 0.09
PC16 0.74 0.85 0.15 0.06
PC17 0.75 0.85 0.04 0.13
PC18 0.72 0.80 0.06 0.27
PC19 0.55 0.65 0.35 0.08
PC20 0.77 0.87 0.1 I 0.08
PC2 1 0.73 0.81 0.09 0.26
PC22 0.80 0.85 0.10 0.21
PC23 0.71 0.76 0.16 0.33
PC24 0.82 0.85 0.05 0.32
PC25 0.77 0.77 0.02 0.4 1
Eigenvalue 14.91 1.35 1.10
Percentage of
variance explained 59.7% 5.4% 4.4%
alpha = 0.98, and for the first administration it was alpha = 0.97. The stability over time, as
measured by the correlation between the two administrations, was r = 0.86. These results
indicate that QPCPA demonstrates considerably high levels of stability over time and internal
consistency, thus leading us to conclude that the instrument is highly reliable4.
Since the internal consistency emerged as being unusually high. a further investigation was carried out in another
organizational context in order to check replicability of this finding. Undergraduate students. participating in a research
course in Industrial and Organizational Psychology under the direction of Aharon Tziner, administered the QPCPA to
28 randomly selected raters in a bank and a fast-food chain. both located in the Montreal area. The 28 raters were similar
to the 157 raters in their socio-demographic characteristics.
It should be noted that the fast-food chain approved the use of a shorter version of the QPCPA, containing only I 1
out of the total number of items (the 1 I items were chosen at random). This reflected the view of the chain executives that
responding to all 25 items of the questionnaire is too time-consuming.
The mean total score on the QPCPA scale were 60.46 and 30.67; the standard deviations were 28.14 and 10.36, and the
Cronbach alphas were 0.97 and 0.83 for the respondents of the Fast-food chain and bank respectively. The difference
between the two alpha coefficients can probably be accounted for by the fact that the lower alpha emanated from the
shorter (reduced) form of the QPCPA. Both figures, are. however, entirely compatible with the previous internal
consistency found in the previous investigation which used the QPCPA with 157 raters.
POLITICAL CONSIDERATIONS I N PERFORMANCE APPRAISAL 187
Convergent and discriminant validities

Convergent validity
The next analysis was an examination of convergent and discriminant validities of the
instrument.
The classical approach to examining convergent validity requires that a new measure of a
construct ought to covary with prevalent measures that purport to gauge the same construct or,
alternatively, with measures of theoretically connected constructs. Pursuing this avenue. we
compared the QPCPA total score with those emanating from the Need for Power Scale from the
manifest Needs Questionnaire (Steers and Braunstein, 1976) and the Machiavellianism
(Mach IV) scale (Christie and Geis, 1970). We judged that these two scales were able to
adequately measure concepts considered to be related to our notion of political considerations in
performance appraisal.
A high need for power (we translated the Need for Power Scale into French using the back-
translation method for present study) is manifested in attempts to achieve control and to
influence other people. The influence attempts serve as a means to determine or get access to
important outcomes. Consequently, we had presumed that a positive correlation could be
expected between QPCPA and need for power, since among the political considerations
perceived as affecting performance appraisal incorporated in the QPCPA, one can find such
statements as: ‘Supervisors produce accurate performance ratings only to the extent that they
may be rewarded for doing so or penalized for failing to do so’, ‘Supervisors give performance
ratings that will make them look good to their superiors’, or, ‘Superiors inflate performance
ratings in order to maximize rewards offered to their employees’. And, indeed, the QPCPA score
did correlate 0.23 (p<O.OI) with the need for power score ( M = 4.46, S.D. = 0.76, alpha = 0.57).
A high score of Machiavellianism (we translated the Mach IV Scale into French using the
back-translation method), as obtained with the Mach IV Scale, has ordinarily been associated
with a salient tendency to manipulate other people and to initiate and control the structure of
interpersonal relationships (Christie and Geis, 1970). We had expected to reveal a positive
correlation between the Machiavellianism score and the QPCPA score, inasmuch as the latter
incorporated responses to statements illustrating attempts to manipulate others, for example:
‘Supervisors give higher performance ratings than deserved in order to gain support/cooperation
from their employees’, ‘Supervisors give low performance ratings to teach a rebellious employee
a lesson’, or, ‘Supervisors give higher performance ratings than is deserved in order to repay
favors to their employees’. The empirical results have indeed substantiated our prediction, as a
reasonable high correlation (Y = 0.33) was found between the QPCPA score and the
Machiavellianism score ( M = 5.07, S . D . = 0.63, alpha = 0.69).
Discriminant validity
To ascertain the extent to which our instrument exhibits discriminant validity, the Organiza-
tional Commitment Questionnaire and the State-Trait Anxiety Inventory were used. We opted
for these two scales, as in our view, they gauge concepts not at all theoretically related to the
concept underlying the QPCPA. This choice conforms to the essential requirement of the
classical procedure for demonstrating that a new instrument has discriminant validity.
The Organizational Commitment Questionnaire (OCQ) (Mowday, Porter and Steers, 1982)
assesses the degree of worker loyalty, satisfaction and pride in the organization as well as
intention to leave. The organizational commitment score ( M = 5.08, S.D. = 0.93, alpha = 0.86)
derived from the French version of the OCQ translation and validation by St-Pierre (1986),
188 A. T Z I N E R ET AL
correlated moderately and negatively with the score of QPCPA, r = -0.27 (p<O.OI)’, thereby
implying that raters who score high on organizational commitment are perceived as being less
inclined to invoke political considerations while formulating their performance Wings6. This
may not be considered as a line of evidence in support of discriminant validity of the QPCPA.
We have further investigated the discriminant validity by correlating the QPCPA score and the
score of State-Trait Anxiety Inventory (see Spielberger, Gorsuch and Lushene, 1979; for a report
of the French version’s psychometric properties see Bergeron, Landry and Belanger, 1976). The
latter scale provides an index score of anxiety. It should therefore not covary at all with the score
of QPCPA. since these two measures have no theoretical common ground.
And, indeed, a correlation of0.14 (n.s.) was found between the score on the STAI ( M = 1.57,
S.D. = 0.35, alpha = 0.86) and that of QPCPA. This result is sufficiently solid to confirm the
discriminant validity of the QPCPA.
Discussion
This paper has set for itself two purposes: (1) to describe the construction process of the QPCPA,
an instrument designed to measure perceptions of the extent to which performance appraisals are
affected by organizational political considerations; and (2) to report the results stemming from
the validation of this tool. Overall, this research has conclusively shown that the instrument can
be considered reliable both with regard to the criterion of internal consistency and to that of
stability over time. Both the internal consistency of 0.97 and the stability of 0.86 induce us to
believe that the QPCPA is highly reliable. The means, standard deviations and the item-total
correlations of the instrument items revealed at Phase 1 exhibit an impressive similarity to those
obtained at Phase 2, thus bolstering the same conclusion. Moreover, the empirical data have
sustained both the convergent and discriminant validities, though in a much less spectacular way.
Probably additional concepts should be used in future investigations for the purpose of
revalidation (e.g. propensity to behave politically, for convergent validity purposes, and a totally
unrelated concept. such as spatial ability instead of organizational commitment’, for
discriminant validity). Furthermore, generalization should be tested with subjects possessing
different characteristics (e.g. belonging to the English speaking milieu, and to different
organizations). Also, larger samples should be obtained so that it would be feasible to conduct a
more complex corroborative analysis (i.e. LISREL) of organizational political considerations
underlying structure. However, the new instrument has opened the door, both to the
investigating in a systematic and quantitative manner the way in which perceived political
considerations affect the psychometric properties of performance ratings as well as to the study
A similar trend also emerged in a study conducted by undergraduate students under the supervision of Aharon Tziner,
as part of their research course requirements.
Based on the responses of 15 raters, randomly selected from a fast-food chain. a measure of organizational
commitment ( M = 3.25: S . D . = 1.21: alpha = 0.82) was found to correlate r = -0.67 (p‘O.01) with the QPCPA score.
‘ As pointed out by an anonymous reviewer, since our study has not been an experimental one. we cannot rule out that
the plausibility of the reverse causal direction; specifically, that organization political considerations might affect one’s
commitment to the organization. Future investigations should ascertain the direction of causality.
’ We thought that organizational commitment would not correlate with our measure of pcwc,priorrs of political behavior
in appraisal, hut it did. In retrospect, we speculate that this might be because raters committed to the organization were
perceived as less likely to be inclined to invoke organizational politics in their Performance appraisals. Exploration of
this possibility is beyond the scope of this paper, and is therefore left for future research.
of t h e distal and proximal factors which predispose raters t o invoke political consideration while
appraising performance. This constitutes a major advancement, since until now studies have
dealt with this m a t t e r in a speculative and qualitative way. The construction and validation o f the
QPCPA should definitely spur research interest in this area.
Acknowledgements
The a u t h o r s t h a n k Sylvie M a r c h a n d , Sophie St-Cyr, a n d Stephan Fournier for assisting in the

d a t a collection and analysis. We also wish t o t h a n k Cecilia Falbe (SUNY a t Albany) for her help
in the formulation and the implementation of this study. T h i s study was supported by a grant
from the Social Sciences and Humanities Research Council of Canada.
We would also like t o express o u r gratitude to t w o anonymous reviewers and t o Dr Daniel C.
G a n s t e r for their insightful comments on the earlier versions of this article.
References
Altman, S., Valenzi, E. and Hodgetts, R. M. (1985). Organizational Behavior: Theory and Practice,
Academic Press, Orlando, FL.
Banks, C. G . and Murphy, K. R. (1985). ‘Toward or narrowing the research-practice gap in performance
appraisal’, Personnel Psychology, 38, 335-345.
Bedeian, A. G . and Zammuto, R. F. (1991). Organization: Theory and Design, The Dryden Press,
Hinsdale, Ill.
Bergeron, J., Landry, M . and Bilanger, D. (1976). ‘The development and validation of a French form of
the State-Trait Anxiety Inventory’. In: Spielberger, C. D. and Diaz-Guerrero, R. (Eds) Cross-Cultural
Anxiety, Halsted Press/Wiley, New York, pp. 41-50.
Bernardin, H. J. and Villanova, P. (1986). ‘Performance appraisal’. In: Locke. E. (Ed.) Generalizing fron7
Laboratory to Field Settings, Lexington, Mass., pp. 43-62.
Bjerke, D. G., Cleveland, J. N., Morrison, R. R. and Wilson, W. C. (1987). ‘Officer fitness report
evaluation study’. Navy Personnel Research and Development Center Report, T R 88-4.
Campion, M. A., Cheraskin, L. and Stevens, M. J. (1994). ‘Career-related antecedents and outcomes ofjob
rotation’, Academy of Management Journaf, 37, 1518-1542.
Christie, R. and Geis, F. L. (1970). Studies in Machiavellianism, Academic Press, San Diego, CA.
Cleveland, J. N. and Murphy, K. R. (1992). ‘Analyzing performance appraisal as goal-directed behavior’,
Research in Personnel and Human Resources Management, 10, 121-185.
Cobb, A. T. (1984). ‘An episodic model of power: Toward an integration of theory and research’, Acadenijs
of Management Review, 9,482493.
Daft, R. L. (1986). Organization Theory and Design, West, St. Paul, MN.
Gioia, D. A. and Sims, H . P. (1985). ‘Self-serving bias and actor-observer differences in organization: An
empirical analysis’, Journal o f Applied Social Psyckologjl, 15, 547-563.
Hogan, E. A. (1987). ‘Effects of prior expectations on performance ratings: A longitudinal study’, A c a t l m i ~ .
of Management Journal, 30, 354-368.
Kane, J. F. and Kane, K. F. (1992). ‘Extension and validation of a model for motivated rating error’.
Unpublished manuscript. University of North-Carolina-Greensboro.
Kumar, K. and Beyerlein, M. (1991). ‘Construction and validation of an instrument measuring
ingratiatory behaviors in organizational setting’, Journal of’ Applied Psychology, 76, 619-127.
Longenecker, C., Sims. H. and Gioia, D. (1987). ‘Behind the mask: The politics of employee appraisal‘.
Academy of Management E.recutive, 1, 183-193.
190 A. TZINER ET AL
Longenecker, C. 0. and Gioia. D. A. (1988. Winter). ‘Neglected at the top: Executives talk about executive
appraisal’, Sloriri M m i i p t i i e n t Review. pp. 41-47.
Mayes. B. T. and Allen. R. W. (1977). ‘Toward a definition of organization politics’, A c ~ c l ( ~of’Matiago- ni~
m v i t Rericw,. 2. 672-678.
Mohrman. A. M. and Lawler. E. E. (1983). ’Motivation and appraisal performance behavior’. In: Landry,
F., Zedeck, S. and Cleveland. J. (Eds) Prt;fortnnticr Me(tsitretiimt c o i d T/ieory. Erlbaum. Hillsdale, NJ,
pp. 173-189.
Morrison, W. E. (1994). ‘Role definitions and organizational citizenship behavior: The importance of the
employee’s perspective’. Acadenij. of Mnnriger?ietit Journal, 37. 1543.- 1567.
Mowday, R., Porter, L. and Steers. R. ( 1982). E i ? i p l o ~ ~ ~ ~ r - O r ~ n i ~Linkages:
i ~ a i i o r i A P.sjdiology ojCornmit-
nient, Ahsenteei.aii arid Turnover. Academic Press, New York.
Murphy, K. R. and Cleveland, J . N. (I99 I). Perforinnrice Appraisal: A n Organizational Perspective, Allyn
and Bacon, Needham Heights. Mass.
Pfeffer, J . (1981). Power in Organirufions, Pittman Publishing Co.. Marshfield, MA.
Prince, J . B. (1991). ‘The characteristics of effective performance appraisal meetings: Manager versus
subordinate views’. Paper presented at the Western Academy of Management, Santa Barbara.
Prince, J. B., Lawler, E. E. and Mohrman. A. M. (1991). ‘Manager-subordinate divergence in perform-
ance’. Unpublished manuscript. Concordia University.
Prince, J. B. and Mohrman. A. M. (1989). ‘Appraising friends: Its effect on attribution process and
manager-subordinate performance rating agreement’. Annual Convention of the Academy of Manage-
ment, Washington, D.C.
Pulakos, E. D. (1986). ‘The development of training programs to increase accuracy in direrent rating
tasks’, Orgrtnizationul Behavior and Hitnian Decision Processes, 38, 76-9 1.
Spielberger, C. D., Gorsuch, R. L. and Lushene, R. E. (1970). Manual ,for the State-Trait An.xiety
Inventory, Consulting Psychologists Press, Palo Alto.
Steers, R. M . and Braunstein, D. N. (1976). ‘A behaviorally based measure of manifest needs in a work
setting‘, Journal of Voc;ational Beliavior, 9, 25 1-266.
St-Pierre. M.-L. (1986). ’Equivalence linguistique et qualites psychomktriques de la traduction franGaise de
I’instrument “Organizational Commitment Questionnaire” (OCQ)’. Unpublished MA thesis, Universitk
de Montreal.
Strahan, R. and Gerbasi, K. C. (1972). ‘Short, homogeneous version of Crowne-Marlow Social Desira-
bility Scale’. Journal of Clinical Psjdiologj., 28, 191-193.
Tushman, M. L. (1977). ‘A political approach to organizations: A review and rationale’, Academy of
Managenietir R(vie,c.. 2, 206-21 6.
Wexley, K. N. and Klimoski, R . (1984). ‘Performance appraisal: An update’. In: Rowland, K. & Ferris, G .
(Eds). Researcli in Personnel and Hitman Resoirrces (vol. 2 ) . Greenwich, CT: JAI Press
Wright, J.. Richer, C., Lecours. A. and Boucher. C. (1987). ‘La traduction de I’echelle de desirabilite
sociale’. Unpublished manuscript. Universite de Montreal.

Development and Validation of A Questionnaire For Measuring Perceived Political

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Development and Validation of A Questionnaire For Measuring Perceived Political

Uploaded by

Copyright:

Available Formats

J O U R N A L OF ORGANIZATIONAL BEHAVIOR, VOL.

17, 179-190 (1996)

Development and validation of a

CCC 0894-3796/96/020 I 79 -1 2 Rewivrtl IS Mnrcli lYY4

Phase 1: preliminary analysis

Phase 2: structure, reliability and validity analyses

28. Supervisors give equivalent performance 2.59 I .70 0.72 0.08

PC 1 0.68 0.60 0.55 0.09

Convergent and discriminant validities

The a u t h o r s t h a n k Sylvie M a r c h a n d , Sophie St-Cyr, a n d Stephan Fournier for assisting in the

You might also like