Cleveland State University

ESC 720
Writing in Electrical and Computer Engineering

Peer Review

Dan Simon

1
 Peer Review
Outline
1. Overview
2. How to conduct a peer review
3. Is the peer review process unethical?
4. Does the peer review process enforce orthodoxy?

2
 Peer Review
What is peer review?
• Journal or conference editor receives a submitted paper
• Editor performs initial check for quality
• Editor sends paper to a few experts for review
• Editor receives reviews and makes a decision
– Accept
– Reject
– Modify (does not apply to conferences due to tight timeline)
3
 Peer Review
Why should you be a peer reviewer?
• Moral obligation for those who submit papers
• Opportunity to see cutting-edge research
• Opportunity for networking

4
 Peer Review
Are you an appropriate reviewer?
• Objectively evaluate your familiarity with the subject

5
 Peer Review
Outline
1. Overview
2. How to conduct a peer review
3. Is the peer review process unethical?
4. Does the peer review process enforce orthodoxy?

6
 Peer Review
How to conduct a peer review:
Is the research novel?
– Literature review is needed
– Sometimes similar results are published simultaneously
– Novelty is a gray area
• What about dissertations?
• What about foreign-language publications?

7
 Peer Review
How to conduct a peer review:
Is the paper clear and logical?
– Is there enough detail? Is there too much detail?
– Is the research reproducible?

8
 Peer Review
How to conduct a peer review:
Is the research significant?
– Is it more than just an engineering exercise?
– Is it important for researchers in the field?
– Is it interesting?
– Is it non-obvious?
– These are difficult questions because they entail value judgements

9
 Peer Review
How to conduct a peer review:
What is the contribution of the paper?
• Is the contribution clearly stated in the abstract, introduction, and
conclusion?
• Are the claims supported in the paper?
• Different types of contributions
• New theory
• New synthesis
• New application
• Tutorial

10
 Peer Review
How to conduct a peer review:
• Recommend for or against publication
• What are the standards of the journal / conference?
• Do you recommend a different publication venue?
• Revision (major, minor)? Resubmission? Is another review needed?
• Justify your review with comments
• Constructive criticism
• General comments
• Specific comments
• For the editor: How confident are you in your review?

11
 Peer Review
How to conduct a peer review:
• Should the qualifications (good or bad) of the author
be considered?
• You need to find a balance between overly permissive
and overly restrictive
• Consider the standards of the target publication
• Students tend to be overly permissive in reviews
• Be diplomatic in your criticism

12
 Peer Review
How to conduct a peer review:
Ethical Issues
• If you submit, then you should review
• Do not review a paper if you are not qualified
• Make sure you disclose any conflict of interest
• Prior publication (conference vs. journal)
• Simultaneous submission
• Plagiarism
• Submitted material is for review only unless the author
allows its use for other purposes
• Do unto others as you would have them do unto you
13
 Peer Review
Outline
1. Overview
2. How to conduct a peer review
3. Is the peer review process unethical?
4. Does the peer review process enforce orthodoxy?

14
 Peer Review
Is the peer review process ethical?
Two different types of peer review:
• Blind
• Double Blind
Most peer reviews are Blind – a few are Double Blind

Proposition: An institution is unethical if it enables and

protects unethical actions

15
 Peer Review
A reviewer was incompetent 62%
A reviewer was biased 51%
A reviewer required unnecessary references to his/her publications 23%
Comments from reviewer included personal attacks 18%
A reviewer delayed the review so he could publish an article on the same topic 10%
A reviewer breached confidentiality 7%
A reviewer used your material without your permission 5%

D. Resnik, C. Guiterrez-Ford, and S. Peddada, "Perceptions of Ethical Problems with Scientific Journal Peer
Review: An Exploratory Study,” 2008
16
 Peer Review
• Who is peer reviewing your papers?
• Not collaborators – they are prevented due to conflict of interest
• Competitors
• The peer review process allows your competitors to anonymously
undermine your work
• The anonymity of peer reviewers is protected
• The process assumes that peer reviewers will act ethically
• An institution is unethical if it enables and protects unethical actions

17
 Peer Review
• What features of peer review make it ethical or unethical?
• Single blind review?
• Double blind review?
• Single blind partially open review? (Philica)
• Open review?
• Reverse single blind review?

18
Peer Review
Outline
1. Overview
2. How to conduct a peer review
3. Is the peer review process unethical?
4. Does the peer review process enforce orthodoxy?

19
 Peer Review
Does peer review enforce orthodoxy?
• Rosalyn Yalow, 1977 Nobel Prize in Physiology or Medicine:
“In 1955 we submitted the paper to Science. … The paper was held
there for eight months before it was reviewed. It was finally rejected.
We submitted it to the Journal of Clinical Investigations, which also
rejected it.”

20
 Peer Review
Does peer review enforce orthodoxy?
• Mitchell Feigenbaum, pioneer of chaos theory:
“Both papers were rejected, the first after a half-year delay. By then,
in 1977, over a thousand copies of the first preprint had been
shipped. This has been my full experience. Papers on established
subjects are immediately accepted. Every novel paper of mine,
without exception, has been rejected by the refereeing process. The
reader can easily gather that I regard this entire process as a false
guardian and wastefully dishonest.”

21
 Peer Review
Does peer review enforce orthodoxy?
• Tuzo Wilson, developed the theory of Hawaiian island formation:
“I … sent [my paper] to the Journal of Geophysical Research. They
turned it down. … They said my paper had no mathematics in it, no
new data, and that it didn’t agree with the current views. Therefore, it
must be no good. Apparently, whether one gets turned down or not
depends largely on the reviewer. The editors, too, if they don’t see it
your way, or if they think it’s something unusual, may turn it down.”

22
 Peer Review
Does peer review enforce orthodoxy?
• Stephen Hawking
• John Bardeen, two-time Nobel prize winner
• Theodore Maiman, inventor of the laser
• … and many others …

23
 Peer Review
Why does peer review enforce orthodoxy?
• Dilution of research talent
• In 1900, 10% of physicists were nominated for Nobel prize
• The number of physicists has increased by a factor of 1,000
• University dependence on grant funding (overhead costs about 50%)

24
References
• “The Task of the Referee,” by Alan Jay Smith
• “A Student’s Guide to Peer Review,” by Dennis Bernstein
• “Is peer review unethical?” by Valentine Cawley

25