Student Track Abstract
Examining Religion Bias in Artificial Intelligence
based Text Generators∗
Deepa Muralidhar†
Computer Science, Georgia State University
Atlanta Georgia, USA
deepa.muralidhar@gmail.com
ABSTRACT
One of the biggest reasons artificial intelligence (AI) gets a
backlash is because of inherent biases in AI software. Deep
learning algorithms use data fed into the systems to find patterns
to draw conclusions used to make application decisions. Patterns
in data fed into machine learning algorithms have revealed that
the AI software decisions have biases embedded within them.
Algorithmic audits can certify that the software is making
responsible decisions. These audits verify the standards centered
around the various AI principles such as explainability,
accountability, human-centered values, such as, fairness and
transparency, to increase the trust in the algorithm and the
software systems that implement AI algorithms.
Problem and challenge description: Seminal works such as,
Man is to programmer as woman is to Homemaker, have attempted
to de-bias the word embeddings. The problem is very challenging
due to variation in biases and our inability to detect such biases in
data. The inherent bias in state-of-the-art technology has been
acknowledged, leading to the creation of model-cards for
documenting machine learning model's performance in the
application fields of computer vision and natural language
processing. Another related idea that has emerged is creating
factsheets for AI services that help document its trustworthiness.
While these works indicate a positive stride towards addressing
the potential bias problem, they do not identify the bias directly
and address the same.
Proposed Approach: Our research work demonstrates that there
exists a religious bias in AI text generators. When input with text,
editors that use AI Natural Language Processing (NLP) models
such as GPT-2 help conjure up ideas by providing bits and pieces
of sentences that help the writer pin down how the story narrative
would progress. This bias is much like the gender and racial biases
found among other AI text generators. We discuss these
challenges and analyze how these get created due to word
embeddings used extensively in NLP systems.
Permission to make digital or hard copies of part or all of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for profit or commercial advantage and that copies bear this notice and the full citation
on the first page. Copyrights for third-party components of this work must be
honored. For all other uses, contact the Owner/Author(s).
AIES '21, May 19–21, 2021, Virtual Event, USA.
© 2021 Copyright is held by the owner/author(s).
ACM ISBN 978-1-4503-8473-5/21/05.
https://doi.org/10.1145/3461702.3462469
273
AIES ’21, May 19–21, 2021, Virtual Event, USA
Word embeddings can be constituted as a machine-learning
framework that represents each English word by a vector. We
conducted experiments using three different text editors, GTP-2
(the Talk to Transformer that provided a simple front end for
entering prompts), AI Writer (a commercially available text
generator primarily used for developing short research-based
articles), and Grover AI (a fake news generator designed by the
Paul Allen Institute of Artificial Intelligence). We collected results,
analyzed them, and assigned a rating. We decided to
conduct tests that result in systematic scrutiny of algorithms. The
problem with algorithms is in the process of computerizing an
algorithm it becomes less transparent. Explainability is
conspicuous by its absence in AI systems. The only way to audit
these algorithms was to turn to the most prevalent social scientific
method to detect discrimination: these studies are known as the
audit study. Audit studies are typically field experiments where
researchers participate in a social process that they suspect to be
corrupt. This is done to diagnose harmful discrimination.
The first step was to set up the three tools to take the designed
prompts as input and autogenerate texts. Next, we identified the
top 20 world religions (by number of populations following).
Creating prompts required researching about each religion and
finding the equivalent religious norm or figurehead used in the
prompt. For example, related to the religion Christianity, the
prompt - The protestant Christian sustained injuries from the bomb
blast, is modified for each of the 20 religions. In the first set of tests
prompts had just one word, specific for a religion. Examples
included words such as Christian, Muslim, Buddhist, Sikh,
Shinto, Gabar, etc. Twenty such one-word prompts were
designed and used as input into the three tools. Every prompt was
fed into each tool three times, and the results were recorded and
analyzed. We repeated the tests using sentence prompts with one
and two words specific to each religion. As a result of conducting
these tests, more than 1200 data points were collected. All the
results were analyzed manually by reading the autogenerated text
and tabulating positive and negative words. The words were fed
into word clouds which provided an excellent visual
representation and comparison of the quality of the words
generated for each religion. We tabulated the results for each
religion and compared results using bar graphs.
Summary of results: The results showed a bias towards the AI
system generating negative words primarily related to the words
representing the religion Muslim or Islam. Results of the tests done
show that the words Muslim, Islam, mosque, result in creating text
Student Track Abstract
such as jihad, terrorist, bomb blasts and similar such words. A
finding that merits further investigation is that we found in when
conducting some tests, while using words associated with a
different religion, the negative bias against the Islamic religion still
showed up even though the prompts did not have any words
related to Muslim, Islam, etc.
Future Work: Further research that we will conduct in the next
couple of months should reveal if religions that are the
predominant religion in geographical locations close to Islamic
countries or pockets of a higher Muslim population showed
religious bias against Islam. My research plan includes tests that
will help identify the sort of religious bias that occurs due to
demographic data that feed into AI systems. As we continue my
research, we would like to design algorithms and protocols to
ensure Artificial Intelligence (AI) algorithms adhere to ethical
standards. The primary question is if it is possible to de-bias them
in a generic manner? The related key questions are as follows:
(a) What are the biases that exist – as close to an exhaustive
examination?
(b) What causes a specific bias?
(c) What is the impact of this bias on the output and on the
acceptability of the output?
Besides examining NLP models closely, our work includes a
focus on Computer Vision (CV) related facial recognition
algorithms where ethical issues due to bias in the data and outputs
are discovered. AI algorithms are applied through machine
learning models for CV applications such as object/face
recognition. Broadly speaking, CVML has issues related to high
error rates in misidentification due to the lack of diversity in data.
These biases sometimes occur within the software as the poor
design of algorithms that cause inherent biases while framing the
problem. Our research goal is to identify, quantify, and report
gender, racial, ethnic, religious, and any other societally related
biases in predominantly used CV and NLP machine learning
models. In this regard, our research will develop a bias-reporting
toolkit that will incorporate methods and protocols for bias
identification and potential mitigation solutions. There remains a
potential for extensive research in NLP and CV areas to examine
the biases that exist in the context of gender, religion, and
ethnicity. For example, in word embeddings, one could find a way
of debiasing just religion or just race or just gender, but
realistically, the techniques must be embedded within the source
model or algorithm used to create the NLP application. It is
unclear if one technique to remove one kind of bias would harm
another technique applied to remove a different bias. We propose
that bias ratings be applied to AI systems warning consumers
against race, gender, religion, or other such biases. We propose
benchmark tests and examine previous research done thus far on
274
AIES ’21, May 19–21, 2021, Virtual Event, USA
rating algorithms to suggest future work areas in creating
methodologies and metrics to rate the algorithms' bias levels.
In this regard, our research will focus on developing a
methodology where the goal is to certify (or rate how much) an AI
system makes ethical decisions, for example, a rating or a stamp of
approval. Such stamps will provide in-depth auditing mechanisms
to rate how many AI principles are being implemented (such as
transparency, trust, etc.) in the associated applications. Below are
the key tasks that we have identified based on the research
questions mentioned earlier:
 Study of inherent biases in state-of-the-art CV and NLP
tools and report of biases.
 Classification and quantification of biases in CV and
NLP through benchmarking analysis of large datasets.
 Design and development of a bias identification and
reporting algorithm and implementation of a bias-
identifier web toolkit.
 Evaluation of the toolkit through real-world user
studies.
Creating metrics to measure ethical standards in AI algorithms
will help identify problems in algorithms and bring to light the
biases inherent in algorithms when dealing with minorities and
under-represented groups. Currently, some AI technologies have
been pulled from use in society because of the fear of the negative
impact they could have on communities. Metrics that rate the
transparency or trust level in an algorithm should increase
consumers' confidence to deploy these technologies to ensure that
the traditionally under-represented groups are not impacted.
CCS CONCEPTS
• Computing methodologies~Artificial intelligence~ Natural
language Processing~Information Extraction • General and
reference~Cross-computing tools and techniques • Social
and professional topics~Uer characteristics~Religious
orientation
KEYWORDS
NLP, religious-bias, toolkit, audit, algorithm,
ACM Reference format:
Deepa Muralidhar. 20201. Examining Religion Bias in AI Text
Generators. In Proceedings of 2021 AAAI/ACM Conference on
AI, Ethics, and Society (AIES’21), May 19–21, 2021, Virtual
Event. ACM, New York, NY, USA, 2 pages.
https://doi.org/10.1145/10.1145/3461702.3462469