Written Analysis for Case Study 1: Therac-25 -
When Code Meets Human Lives
Abstract — The tragic string of events the surrounds the
Therac-25 between 1985 and 1987 lead to the loss of at least six
lives. Only holding the programmer responsible for this would
be a misjudgment since multiple people, directly or indirectly,
contributed to the resulting tragedy. The tragedy is a prime
example that demonstrates three important ideas: firstly, the
manufacturer is ultimately responsible for the quality and
reliability of the software in their products; secondly, end-users
also contribute to software and hardware failure; and lastly,
user feedback is an essential part of software improvement.
Keywords —Therac-25, Sociotechnical system, corporate
decisions, cutting corners
I. INTRODUCTION
Cancer treatment, just like the deadly disease itself, must
be treated with utmost importance. Given that the treatment
involves exposure to radiation, the room for error is slim to
none. Therac-25 was meant to be a revolutionary method of
treatment that utilised a “double-pass” concept for electron
acceleration. Due to manufacturer neglect and design flaws
in software and hardware, the technology ended up taking
the lives of at least 6 people by administering lethal doses to
patients between 1985 and 1987. This analysis of the events,
based on the subsequent investigation published by Nancy
Leveson and Clark Turner, expands on three ideas that
attempt to support the theory that only holding the
programmer responsible for this would be a
misjudgment since multiple people, directly or indirectly,
contributed to the resulting tragedy.
II. SUPPORTING ARGUMENTS
A. The manufacturer is ultimately responsible for the
quality and reliability of the software in their products.
A company like AECL hires engineers, programmers,
medical specialists, and various other people to work for
them, and while they are given the responsibility of
producing certain results, they are not responsible for the
final product. The Therac-25 was built over many years and
iterations by a multitude of people working at AECL. Those
who worked on it indeed developed the puzzle pieces that
ultimately built the Therac-25, but the final product, with all
its flaws, belongs to AECL. As a programmer myself, the
fact that only one programmer was hired was an instant bad
idea. As the sole programmer, they would have no
colleagues to bounce ideas off of, or to verify each other’s
existing code. Assembly language being an inherently more
difficult, with more opportunity for errors, would almost
force programmers to generate documentation for their
colleagues’ and their own use. The decision to hire only one
programmer to write all the code, from the Therac-6 to the
Therac-20, and finally to the Therac-25, was not the
programmer’s decision. Neither was the decision to remove
the hardware locks on the Therac-25. It could even be
argued that AECL was to blame for very little
documentation being made during development as the ones
in charge, they are to set guidelines for software
development and enforce them. This could be compared to
the data breach of Facebook and Cambridge Analytica. At
least 87 million accounts had been affected during the
breach [1]. Another case was the Y2K bug of 2000. The
primary cause behind this was the fact that computer storage
was expensive, so only the last two digits were used to store
the year. The concern was that when the year 2000 would be
reached, computers would read that as 1900 [2]. This
affected banks, businesses, power plants, among others. In
both cases, blaming the programmers would be arguably
wrong since they were limited by hardware capacities. As
for the data breach, developers of both Facebook and
Cambridge Analytica are largely bound by the tasks
assigned to them by others who are higher up in the chain.
Corporate decisions like cost-cutting or business malpractice
are not taken by developers or engineers, they are taken by
owners and board members of said company. Thus, as the
owners and manufacturers of the Therac-25, AECL must
bear the brunt of responsibility.
B. End-users also contribute to software and hardware
failure
The Therac-25 was a terribly designed tool that was also
used carelessly by its users. The most harrowing example of
this is the fact that error messages were ignored by the
operators on a regular basis. Upon seeing the “H-tilt” and
“no dose” on the system display, the operator simply
proceeded with the treatment. Another account led to the
machine shutting down after displaying “Malfunction 54”.
Prior to this, the operator has mistakenly typed “x” instead
of “e” for the required treatment. According to the
investigation, the operator had become accustomed to typing
“x” for administering X rays, but it is not unreasonable to
expect that an operator in charge of such machine would pay
more attention to what they type, rather than what they are
used to typing. Furthermore, the only way the patient in the
shielded room could communicate with the operator was
through a video and audio monitor, both of which were out
non-functional at the time. This is completely out of the
control of both the programmer and even the manufacturer.
The hospital, being the end-user in a sense, is responsible for
the video and audio monitors being in working order.
C. User feedback is essential and must not be neglected
The Sociotechnical theory states that social practices,
social relationships, and social institutions are required to
design, produce, distribute, and use technology [4]. One
interpretation of this could be that technology can only be
developed and improved based on the people that use it, and
how they use it. In the context of the Therac 25, it had
people using it and it also had people who found it useful,
but that was it. AECL made no attempts to investigate the
concerns of the very people who made the technology useful
in the first place. Multiple times, AECL blatantly refused to
acknowledge or investigate reports of possible overdoses.
They simply claimed that it was impossible. We now know
that the amount of testing done during development was
nowhere near adequate, so AECL’s confidence on the
Therac-25 was grossly unfounded. This was not a form of
technology that served a relatively harmless use, like a
calculator or an office computer. This was a machine that
had already taken lives by the time any semblance of an
investigation took place. Both the FDA and AECL were
extremely slow to act, and worse yet, kept the machines
operational and continued treating patients. This can be
interpreted as a form of corporate greed, prioritizing sales,
and cutting costs above the lives of the patients. The
programmer could definitely be held responsible for the lack
of testing and documentation, but without AECL’s approval,
they could not have acted on user feedback if they wanted to
fix the flaws in his software.
III. CONCLUSION
The tragedy was series of events in which the
operators, manufacturer and the programmer all carry
the blame, and thus, the programmer cannot be the only
one held responsible. They are definitely to blame for the
lack of documentation, testing and safety oversights in his
software. Despite his faults, had AECL enforced more rigid
software development practices, hired more programmers,
not cut corners when developing the machine, it could be
argued that none of this would have happened. The decision
to refute any reports of overdose was also in the hands of
AECL, not the programmer. The analysis of this case
hopefully conveys the message that being overly cautious,
especially when dealing with the life-saving treatments, is
far more important than being overly concerned with
revenues and margins.
ACKNOWLEDGMENT (Heading 5)
I’d like to thank Professor Christina Penner for
introducing me to this case study and ethics regarding
computer science as a whole. Furthermore, I’d also like to
thank my group members from Group #7 who offered
invaluable ideas during the peer review phase. Finally, I’d
like to thank John from the YouTube channel Plainly
Difficult, who humorously but accurately portrayed the
events surrounding the Therac-25 in an animated short
documentary.
REFERENCES
[1] I. Sherr, “Facebook, Cambridge Analytica, Data Mining and trump:
What you need to know,” CNET, 18-Apr-2018. [Online]. Available:
https://www.cnet.com/news/facebook-cambridge-analytica-data-
mining-and-trump-what-you-need-to-know/. [Accessed: 17-Feb-
2022].
[2] National Geographic Society, “Y2K Bug,” National Geographic
Society, 09-Oct-2012. [Online]. Available:
https://www.nationalgeographic.org/encyclopedia/Y2K-bug/.
[Accessed: 17-Feb-2022].
[3] N. G. Leveson and C. S. Turner, "An investigation of the Therac-25
accidents," in Computer, vol. 26, no. 7, pp. 18-41, July 1993,
doi:10.1109/MC.1993.274940.
[4] D. G. Johnson, “Democracy, technology, and Information Societies,”
The Information Society: Innovation, Legitimacy, Ethics and
Democracy In honor of Professor Jacques Berleur s.j., pp. 5–16, 2007,
doi:10.1007/978-0-387-72381-5_2.