Professional Documents
Culture Documents
PDF Our Humanity Exposed Predictive Modelling in A Legal Context Stanley Greenstein Ebook Full Chapter
PDF Our Humanity Exposed Predictive Modelling in A Legal Context Stanley Greenstein Ebook Full Chapter
https://textbookfull.com/product/modelling-our-changing-world-
jennifer-l-castle/
https://textbookfull.com/product/the-opposite-of-hate-a-field-
guide-to-repairing-our-humanity-first-edition-kohn/
https://textbookfull.com/product/humanness-as-a-protected-legal-
interest-of-crimes-against-humanity-conceptual-and-normative-
aspects-rustam-atadjanov/
https://textbookfull.com/product/big-data-in-context-legal-
social-and-technological-insights-1st-edition-thomas-hoeren/
Artificial Intelligence Computational Modelling and
Criminal Proceedings A Framework for A European Legal
Discussion Serena Quattrocolo
https://textbookfull.com/product/artificial-intelligence-
computational-modelling-and-criminal-proceedings-a-framework-for-
a-european-legal-discussion-serena-quattrocolo/
https://textbookfull.com/product/the-power-of-human-how-our-
shared-humanity-can-help-us-create-a-better-world-adam-waytz/
https://textbookfull.com/product/the-map-of-meaningful-work-2e-a-
practical-guide-to-sustaining-our-humanity-2nd-edition-lips-
wiersma/
https://textbookfull.com/product/endocrine-conditions-in-
pediatrics-a-practical-guide-takara-stanley/
https://textbookfull.com/product/a-first-course-in-predictive-
control-second-edition-rossiter/
!"#! #$%#&'$( $')''
* +* "#+ # ,$-)
" "" ) ! ""
+ + # ! +
" ! #!)## " "# +
# " # # + "# #
! #)
#! " " " ). # ! # " #
) " # # ""
+"#! ) # #
+ " ! # ) !#
" )
" # +" " # !#
"" # ) " + + +
#" " + + " " ! )
# #! " # +" " # )
# " + /! )
0# " " " #+ # #
" " # )" # + 1 0# "
2#3 40235 * 3# 4*35 " ! 0# "
40 5) # + 6" 7# )
" " #+ " ! "
# ) # " # "8
&'$(
"9::#)!): 8#;#9!99#9 $-$<=(
.>?(1?$(<-?(-1(
.>?(1?$(<-?(-?-
+$'<?$
OUR HUMANITY EXPOSED
Predictive Modelling in a Legal Context
Stanley Greenstein
Our Humanity Exposed
Predictive Modelling in a Legal Context
Stanley Greenstein
©Stanley Greenstein, Stockholm University 2017
I have heard that writing a thesis is like running a marathon. Having run two
marathons I feel that I am in a position to comment. There are similarities in
that both are long, they require lots of training and endurance and they both
have many ups and downs. However, there are some major differences. A
marathon has race officials to show the way, the finish line is clearly demar-
cated and it is not a team effort. This is definitely not the case when writing a
thesis.
I would therefore like to take this opportunity to thank my thesis team. I
would like to start by extending my appreciation to my supervisor, Peter
Wahlgren (Professor in Law and Information Technology), who provided the
legal perspective and who was always available to discuss both legal and tech-
nical developments. A big thank you also to my deputy supervisor Jussi
Karlgren (Adjunct Professor in Language Technology), who provided the
technical perspective and who was able to explain the intricate complexities
of technical phenomena in an understandable manner. Thank you both also
for translating this thesis into Swedish. During the second to last year of writ-
ing, the three of us would meet regularly to discuss what I had written and
plan the way forward. These meetings were insightful in that I experienced
what it is like to review a text with a lawyer on one side and a computer sci-
entist on the other. It was motivating, it was informative, it was frustrating but
most of all it was fun! Thank you!
I would also like to express my gratitude to Cecilia Magnusson Sjöberg
(Professor in Law and Information Technology). Thank you for your patience
over the years for all those occasions I barged into your office without warning
in order to get advice on one matter or another. Most of all thank you for
giving me the freedom and space during these past six months so that I could
focus entirely on completing this thesis. I am fully aware that this could not
have been easy considering your responsibilities for making sure that all lec-
turing and examination duties were covered.
I would like to thank Peter Seipel (Professor Emeritus in Law and Infor-
mation Technology), who took an interest in my project early on and, as a
pioneer in the field of legal informatics, made the path easier for the rest of us,
in this the most interesting and relevant field of law. A special thank you also
to my present colleagues at the Swedish Law and Informatics Research Insti-
tute (IRI): Liane Colonna, Maeve Dion, Mårten Edenroth, Ängla Eklund, Al-
exandra Sackemark, Madeleine Sandberg and Christine Storr. Thank you for
making sure that the daily existence as a doctoral student was not as lonely as
it sometimes is made out to be.
A special thank you also to Mark Klamberg for taking on the role of ‘op-
ponent’ at my final seminar, for providing me with all the feedback and con-
structive criticism, especially within the area of international law and for tak-
ing the time to address all my follow-up queries.
This thesis would not have been possible without the support of the Faculty
of Law, Stockholm University. After receiving the opportunity to embark on
doctoral studies, the Faculty of Law allowed me to pursue my doctoral studies
within my appointment as an Adjunkt, in effect financing this project. Many
thanks also to the Stiftelsen av den 28 oktober 1982 for the generous yearly
book grant, allowing me to purchase the necessary reading material.
Many thanks must also be given to a number of individuals at the Faculty
of Law, Stockholm University. First, my appreciation to Åsa Hellstadius, for
providing suggestions regarding the structuring of this thesis, for assisting me
with the thesis template and most of all for reassuring me that the time was
ripe to proceed with the opposition of this thesis. Also, considering that this
thesis delved into areas of law that I am not a specialist in, I would like to
thank Helene Andersson, Antonina Bakardjieva-Engelbrekt, Claes Granmar,
Jaan Paju, Martin Ratcovich, Pål Wrange, Torben Spaak and Mauro Zamboni.
In addition, many thanks to Bengt Baeckmark, Laura Carlson, Jonas Ekfeldt,
Ulf Färjare, Agita Akule Larsson, Lydia Lundstedt, Grant McWilliams, Dan
Olsson, Panagiotis Papapetrou, Christina Ramberg, Ida Söderberg Tah, Staf-
fan Westerlund and Roger Persson-Österman. You have all helped me in some
way or another and at some point in time. You have all been generous with
your time, support and words of encouragement. Thank you!
A special thank you also to Lee Bygrave of the Senter for rettsinformatikk,
Department of Private Law, Oslo University. We would meet at the annual
Nordic Conference on Law and IT where you would always show an interest
in my progress, offer valuable insights and most importantly not hold back as
far as providing words of encouragement was concerned. Tusen takk!
In addition, the cover of this thesis would be blank were it not for the cre-
ative efforts of Simon Dobrzynski and Indra Jungkvist. Thank you both for all
the time spent on enhancing the graphic impression of this thesis. A huge
thank you also to Laura Chadwick, who on extremely short notice agreed to
proof read my thesis, providing valuable input and improving the overall im-
pression of the text markedly. Your assistance with this thesis was invaluable!
All mistakes in this text should be attributed to changes made by me after it
was proof read. I would also like to thank the staff of the Stockholm University
libarary for helping with the production of this thesis.
On a more personal note, I would like to thank my in-laws, Kalle and Eva-
Lii, for making life a lot easier in many respects since I moved to Sweden.
Many thanks also to Indra and Simon for the warmth you have both shown
over the years. Finally, a special thank you to my wife and soulmate Anna, for
the love and companionship over the years. You have always been positive
about this project even when faced with a partner that has on occasion been ‘a
little less positive’. Thank you for polishing the cover texts and Swedish trans-
lation – when it comes to working with text, you are in a class of your own.
Thank you also for bearing the major load of family life during the run up to
the completion of this thesis. Finally, I would probably not have survived this
excruciating process were it not for my children Levi, Nomie and Jonah. Your
mere existence has constantly put things into perspective and reminded me
that this is just a thesis, nothing more!
Stanley Greenstein
Contents
1 Introduction .................................................................................. 21
1.1 Problem Identification............................................................................. 22
1.2 Overall Objectives ................................................................................... 26
1.3 Research Questions................................................................................. 29
1.4 Method and Material ............................................................................... 30
1.5 Definitions ................................................................................................. 41
1.6 Perspectives ............................................................................................. 48
1.7 Scope and Delimitation .......................................................................... 52
1.8 Previous Research ................................................................................... 55
1.9 Positive Aspects of Predictive Modelling .............................................. 61
1.10 The Dangers Associated with Predictive Modelling ...................... 64
1.11 Structural Overview ........................................................................... 68
AI Artificial Intelligence
AmI Ambient Intelligence
API Application Programming Interface
CJEU Court of Justice of the European Union
CoE Council of Europe
DPD Data Protection Directive
DDD Data-Driven decision-making
EU Charter Charter of Fundamental Rights of the European Union
GDPR General Data Protection Regulation
ECHR European Convention on Human Rights
ECtHR European Court of Human Rights
ICCPR International Covenant on Civil and Political Rights
ICESCR International Covenant on Economic, Social and Cultural
Rights
IoT Internet of Things
IP Internet Protocol
ICT Information and Communication Technology
ISP Internet Service Provider
KDD Knowledge Discovery in Databases
NSA National Security Agency (US)
OECD Organization for Economic Co-Operation and Development
PTS Swedish Post and Telecom Authority
RFID Radio Frequency Identity
TEU Treaty of the European Union
TEFU Treaty on the Functioning of the European Union
TTP Trusted Third Party
UDHR Universal Declaration of Human Rights
UDID Unique Device Identifier
UN iUnited Nations
US United States
1 Introduction
This is a thesis about technology and the law. The realization of the technol-
ogy-centric society, with mobile technologies at the forefront of this develop-
ment, has made the on-line environment accessible to the masses. Due to the
popularity of the applications that can be utilized from within this environ-
ment, their ease of use, and the convenience associated with them, people are
spending a large part of their lives in this environment. Whereas previously
one could refer to the ‘on-line world’ and ‘off-line world’, this distinction is
becoming blurred, as these two spheres continue to merge. The hyper-connec-
tivity characterising present day society is being spurred on by the use of the
digital media, such as the internet and more specifically the social media,
which are increasingly being used not only for everyday means of communi-
cation and commercial purposes, but also as an instrument by means of which
people are publically documenting their lives. There are many advantages as-
sociated with this on-line environment. The huge amounts of data being pro-
duced every second and contributing to the notion ‘big data’ hold great poten-
tial for humanity, with the phrase ‘data-driven innovation’ frequently used in
commercial and technological settings. New technologies are being developed
to exploit big data in order to extract previously inaccessible knowledge from
it.
While the advantages to be gained from the exploitation by new technolo-
gies of big data are undisputed, their use also entails the exposure of individ-
uals and society to risks and vulnerabilities, which multiply as the reliance on
digital technologies increases. In addition, these new technologies are being
incorporated into decision-making systems, which are increasingly relied on
to make unsupervised decisions about humans and facilitate social control.
The risks and vulnerabilities associated with this technology also increase as
these technologies and systems become ambient and automated. Ultimately,
this state of affairs requires a consideration as to what kind of society is de-
sired, and how great a degree of human agency people are willing to acquiesce
21
to technology, in return for the benefits that it brings. Taking this argumenta-
tion one step further, and as society’s reliance on decision-making systems
increases, a concern is whether, in the near future, humans will even be in a
position to have a say in the nature of the fabric of society, human agency
having slowly been eroded as digital outputs determine the basis for social
steering.
The more data modern decision-making systems can use to base their pre-
dictions on, the more accurate their predictions. Consequently, actors using
these technologies have acquired an insatiable thirst for data. This in turn has
resulted in a society characterised by surveillance. This surveillance is made
possible by the architecture of the digital environment, which is essentially
made up of computer hardware and code, and which allows for the tracking
of a person’s every move as they make their way through this coded space.
Individuals’ movements, ideas, preferences, habits, personality traits and
opinions are monitored, recorded and stored for an infinite period of time, to
be used at a later stage for some purpose that was not anticipated at the time
of collection.
Humanity is presently faced with the challenge of addressing the hazards
associated with relying too heavily on the outputs from decision-making sys-
tems without taking into account the pitfalls. The extent to which a positive
outcome can be achieved will depend on the extent to which the relationship
between law, technology and society can be exploited. It is within this context
that predictive modelling forms the object of this legal examination.
The catalyst of this thesis is the challenges posed by the increased reliance on
digital decision-making systems, which incorporate predictive models. It is a
common theme within the field of legal informatics that technology brings
with it both advantages and disadvantages. This is no different as far as pre-
dictive modelling is concerned, which can be described as the incorporation
of machine learning algorithms into models, which learn from historical data
and have the ability to apply this novel insight on new data. The following is
a formal definition of the predictive model:
22
A predictive model captures the relationships between predictor data and be-
haviour, and is the output from the predictive analytics process. Once a model
has been created, it can be used to make new predictions about people (or other
entities) whose behaviour is unknown.1
1 Finlay, Steven, Predictive Analytics, Data Mining, and Big Data: Myths, Misconcep-
tions and Methods, Palgrave Macmillan, 2014, at p. 215.
2 English Oxford Living Dictionaries, Algorithm, available at https://en.oxforddiction-
aries.com/definition/algorithm (last accessed on 2017-03-19).
3 Weinberger, David, The Machine That Would Predict the Future, Scientific Ameri-
can, December 2011 at p. 32.
4 Neal, David T., Wood, Wendy and Quinn, Jeffry M., Habits- A Repeat Performance,
Current Directions in Psychological Science, August 2006, Vol. 15 no. 4, pp 198-202,
at p. 198, available at http://cdp.sagepub.com/content/15/4/198.abstract (last accessed
on 2016-11-25), see also Chi, Kelly Rae, Why are Habits So Hard to Break?,
DukeToday, available at https://today.duke.edu/2016/01/habits (last accessed on
2016-11-25).
23
the greater the extent to which a human action has been repeated in the past,
the greater the probability will be that it will occur in the future. It is on this
premise that algorithms, using machine-learning techniques, search data and
identify examples of human actions, acquiring the ability to assess the proba-
bility of these actions re-occurring in the future.
Predictive models determine a considerable amount of human interaction
with the digital environment: they determine the content that an individual
sees or does not see, they determine what an individual listens to, they deter-
mine who an individual will meet, they determine whether an individual will
be considered for a job or not, they determine whether an individual is a credit
risk, they determine whether an individual is a security risk and they determine
the health status of a person, to mention but a few applications. Ultimately,
predictive models make decisions about humans.
The power emanating from the access to predictive models increases as the
knowledge they identify is combined with the behavioural sciences, which
provide even greater insight into human behaviour. It has been stated that,
‘[w]e live in an age of psychology and behavioural economics – the behav-
ioural sciences’.5 Consequently, with this insight into human behaviour,
comes power, as the ability to identify human behaviour brings with it the
ability to influence and alter human behaviour. Subsequently, people are sus-
ceptible to being manipulated into making choices that they otherwise may
not have made, this manipulation taking place covertly, in turn threatening
personal autonomy and the notion of the individual as an autonomous agent.
Many actors within society are utilizing predictive modelling, for example,
governments, public authorities, law enforcement, the health care sector and
most notably, private commercial actors.6 Sunstein remarks:
For-profit companies are using behavioural research every day. They want to
learn how people think and to use that learning to make money. Charitable
organizations consult behavioural scientists to find out how they might attract
donors and increase donations. For their part, public officials are increasingly
turning to the behavioural sciences to promote their goals. They are influencing
people in multiple ways in order to reduce poverty, to increase employment,
5 Sunstein, Cass R., The Ethics of Influence: Government in the Age of Behavioural Sci-
ence, Cambridge University Press, 2016, at p 1.
6 The term ‘commercial actor’ is used for the most part throughout this thesis. It refers
to private companies or corporations, these terms used interchangeably.
24
to clean the air, to improve health, to encourage people to vote, and to increase
safety on the highways. What are the ethical constraints on their actions? 7
Witten and Frank highlight the connection between predictive modelling and
business:
A scientist’s job … is to make sense of data, to discover the patterns that gov-
ern how the physical world works and encapsulate them in theories that can be
used for predicting what will happen in new situations. The entrepreneur’s job
is to identify opportunities, that is patterns in behaviour that can be turned into
a profitable business, and exploit them.8
25
For all the benefits associated with predictive modelling, there are a num-
ber of potential harms that can negatively affect the individuals upon whom
these models operate. There is little public awareness of this practice, and
when knowledge of its existence is publicised, it results in a feeling of uncer-
tainty, insecurity and a loss of control on the part of individuals, who are the
target of this powerful technology. However, the harms resulting from predic-
tive modelling are not only emotional. Predictive modelling can result in
harms that are concrete in nature and with a varying degree of recognition in
the law. This thesis investigates these potential harms in the context of a spe-
cifically chosen legal framework from within the European legal space, in or-
der to determine to what extent they are addressed. In doing so, it examines
alternative mechanisms that could potentially improve the situation of the in-
dividual in the face of the use of predictive models. The main challenge, there-
fore, is how to level the playing field between commercial actor and individual
in the commercial setting.
This thesis has a number of aims. The first aim is to create a societal awareness
of the existence of predictive modelling. While this aim may seem simplistic
and trivial in nature, it is deemed a necessary step towards protecting society
from the associated dangers. It is also as a result of increased societal aware-
ness that greater demands can be placed on those developing the technology
as well as those operating it, to bring about positive change.
The second aim of this thesis is to identify and make an inventory of the
risks and vulnerabilities associated with the use of predictive modelling. Stud-
ies have shown that both internationally and nationally, there is a limited ap-
preciation for the effects of technologies such as predictive modelling.11 It is
in pursuing this aim that the potential harms resulting from the use of predic-
tive modelling techniques are explored. An inventory of these harms is com-
piled by studying the vast amount of reading material on the topic, both within
law but also within other disciplines, such as information communication
11 Swedish Government Official Reports, SOU 2016:41, Hur står det till med den per-
sonliga integriteten? – en kartläggning av Integritetskommittén, Delbetänkande av In-
tegritetskommittén, Stockholm 2016, at p. 52.
26
technology, sociology, psychology, the behavioural sciences and surveillance
studies.
The third aim of this thesis is to examine if, and to what extent, traditional
law addresses the potential harms resulting from predictive modelling as well
as whether, and to what extent, predictive modelling itself is regulated. This
is achieved by examining the degree to which the European data privacy legal
framework addresses personal autonomy and predictive modelling.12
The fourth aim of this thesis is to provide an alternative strategy to mini-
mize the risks and vulnerabilities connected to predictive modelling and ad-
dress the potential harms. In this regard the strategy of empowerment is intro-
duced. It comprises various components that possess either a traditional legal
character, have a soft law character, are technical in nature or are neutral in
nature.13 It is within this regulatory mechanism that traditional law retains its
role as a regulatory instrument, but where soft law, technology and other com-
ponents are suggested as a complement. The strategy of empowerment also
highlights the role of traditional law as an indirect regulatory instrument.
The fifth aim of this thesis is to demonstrate the complexities that a solution
must take into consideration, in that there are multiple interests at stake. For
example, the interests of the individual may clash with those of society-at-
large. An illustration of this is provided by considering taxation systems,
where predictive models assist the tax authorities in identifying tax evasion.
The excessive empowerment of individuals in this context, for example in the
form of providing knowledge concerning tax evasion detection capabilities,
while empowering individuals, would also allow them to strategically bypass
the system in order to evade detection, a phenomenon referred to as ‘gaming’
a system.14 This may be beneficial from the tax evader’s point of view (indi-
vidual), but is harmful from the societal perspective, as the services provided
12 This thesis does not promote the European legal framework of data privacy as the
only, or even most suitable, framework for addressing the harms associated with this
technology. For example, predictive models or like technologies, could very well be
effectively addressed by other legal frameworks, be they international or national, or
other legal spheres, for example, consumer protection, labour law, competition law or
contract law, to mention but a few.
13 In this context, the label ‘neutral’ refers to the fact that a component does not fall un-
der one of the other labels (traditional law, soft law or technology). For example, the
access to knowledge is one such component.
14 Kroll, Joshua A., Huey, Joanna, Barocas, Solon, Felten, Edward W., Reidenberg, Joel
R.,Robinson, David G., and Yu, Harlan, Accountable Algorithms, Vol. 165, Issue 3,
27
Another random document with
no related content on Scribd:
korppi se istui männyssään, kuin veistos liikkumaton, ja mietti
syvästi ihmeissään: »Mikä Johanssoneilla nyt on?»
Kaunein laulu.
I.
II
III.
Kauneinta on hämärtyissä.
Hopeahuilu.
Kirje. (EV)
Kevätsade. (EV)
Impromptu.
Ja taivaalta tähtiterttu
ihan riippuu poimijan käteen,
nukentalosi ikkunoihin
luo seitsenkertaisen säteen.
Rovio.
Ihmisen koti.
Tähtien lohdutus.
Sataa. (LV)
Updated editions will replace the previous one—the old editions will
be renamed.