LITERATURE SURVEY – AI MINI PROJECT

Shreyans Chhajed 332010 21911107

Group Number - 7

RESEARCH PAPERS
1. Key Point Analysis Via Contrastive Learning and Extractive Argument Summarization
This paper presents the proposed approach to the Key Point Analysis shared task, collocated
with the 8th Workshop on Argument Mining.

Key point analysis is the task of extracting a set of concise and high-level statements from a
given collection of arguments, representing the gist of these arguments. This paper presents our
proposed approach to the Key Point Analysis shared task, collocated with the 8th Workshop on
Argument Mining. The approach integrates two complementary components. One component
employs contrastive learning via a Siamese neural network for matching arguments to key
points; the other is a graph-based extractive summarization model for generating key points. In
both automatic and manual evaluation, our approach was ranked best among all submissions to
the shared task.

2. Privacy Policy Question Answering Assistant: A Query-Guided Extractive

Summarization Approach
Existing work on making privacy policies accessible has explored new presentation forms
such as color-coding based on the risk factors or summarization to assist users with
conscious agreement. To facilitate a more personalized interaction with the policies, in
this work, we propose an automated privacy policy question answering assistant that
extracts a summary in response to the input user query. This is a challenging task
because users articulate their privacy-related questions in a very different language than
the legal language of the policy, making it difficult for the system to understand their
inquiry. Moreover, existing annotated data in this domain are limited. We address these
problems by paraphrasing to bring the style and language of the user's question closer
to the language of privacy policies. Our content scoring module uses the existing in-
domain data to find relevant information in the policy and incorporates it in a summary.
Our pipeline is able to find an answer for 89% of the user queries in the privacy QA
dataset

3. Who Says Like A Style of Vitamin: Towards Syntax-Aware Dialogue Summarization

Using Multi-task Learning
Abstractive dialogue summarization is a challenging task for several reasons. First, most
of the important pieces of information in a conversation are scattered across utterances
 through multi-party interactions with different textual styles. Second, dialogues are
often informal structures, wherein different individuals express personal perspectives,
unlike text summarization, tasks that usually target formal documents such as news
articles. To address these issues, we focused on the association between utterances
from individual speakers and unique syntactic structures. Speakers have unique textual
styles that can contain linguistic information, such as voiceprint. Therefore, we
constructed a syntax-aware model by leveraging linguistic information (i.e., POS
tagging), which alleviates the above issues by inherently distinguishing sentences
uttered from individual speakers. We employed multi-task learning of both syntax-
aware information and dialogue summarization. To the best of our knowledge, our
approach is the first method to apply multi-task learning to the dialogue summarization
task. Experiments on a SAMSum corpus (a large-scale dialogue summarization corpus)
demonstrated that our method improved upon the vanilla model. We further analyze
the costs and benefits of our approach relative to baseline models.

4. Investigating Entropy for Extractive Document Summarization

Automatic text summarization aims to cut down readers’ time and cognitive effort by
reducing the content of a text document without compromising on its essence. EG
informativeness is the prime attribute of document summary generated by an
algorithm, and selecting sentences that capture the essence of a document is the
primary goal of extractive document summarization. In this paper, we employ Shannon
entropy to capture informativeness of sentences. We employ Non-negative Matrix
Factorization (NMF) to reveal probability distributions for computing entropy of terms,
topics, and sentences in latent space. We present an information theoretic
interpretation of the computed entropy, which is the bedrock of the proposed E-Summ
algorithm, an unsupervised method for extractive document summarization. The
algorithm systematically applies information theoretic principle for selecting informative
sentences from important topics in the document. The proposed algorithm is generic
and fast, and hence amenable to use for summarization of documents in real time.
Furthermore, it is domain-, collection-independent and agnostic to the language of the
document. Benefiting from strictly positive NMF factor matrices, E-Summ algorithm is
transparent and explainable too. We use standard ROUGE toolkit for performance
evaluation of the proposed method on four well known public data-sets. We also
perform quantitative assessment of E-Summ summary quality by computing its semantic
similarity w.r.t the original document. Our investigation reveals that though using NMF
and information theoretic approach for document summarization promises efficient,
explainable, and language independent text summarization, it needs to be bolstered to
match the performance of deep neural methods.
5. Recursively Summarizing Books with Human Feedback
A major challenge for scaling machine learning is training models to perform tasks that
are very difficult or time-consuming for humans to evaluate. We present progress on
this problem on the task of abstractive summarization of entire fiction novels. Our
method combines learning from human feedback with recursive task decomposition: we
use models trained on smaller parts of the task to assist humans in giving feedback on
the broader task. We collect a large volume of demonstrations and comparisons from
human labelers, and fine-tune GPT-3 using behavioral cloning and reward modeling to
do summarization recursively. At inference time, the model first summarizes small
sections of the book and then recursively summarizes these summaries to produce a
summary of the entire book. Our human labelers are able to supervise and evaluate the
models quickly, despite not having read the entire books themselves. Our resulting
model generates sensible summaries of entire books, even matching the quality of
human-written summaries in a few cases ($\sim5\%$ of books). We achieve state-of-
the-art results on the recent Book Sum dataset for book-length summarization. A zero-
shot question-answering model using these summaries achieves state-of-the-art results
on the challenging Narrative QA benchmark for answering questions about books and
movie scripts. We release datasets of samples from our model.

6. Email Sum: Abstractive Email Thread Summarization

Recent years have brought about an interest in the challenging task of summarizing
conversation threads (meetings, online discussions, etc.). Such summaries help analysis
of the long text to quickly catch up with the decisions made and thus improve our work
or communication efficiency. To spur research in thread summarization, we have
developed an abstractive Email Thread Summarization (Email Sum) dataset, which
contains human-annotated short (<30 words) and long (<100 words) summaries of 2549
email threads (each containing 3 to 10 emails) over a wide variety of topics. We perform
a comprehensive empirical study to explore different summarization techniques
(including extractive and abstractive methods, single-document and hierarchical models,
as well as transfer and semi supervised learning) and conduct human evaluations on
both short and long summary generation tasks. Our results reveal the key challenges of
current abstractive summarization models in this task, such as understanding the
sender's intent and identifying the roles of sender and receiver. Furthermore, we find
that widely used automatic evaluation metrics (ROUGE, BERT Score) are weakly
correlated with human judgments on this email thread summarization task

7. Decision-Focused Summarization
 Relevance in summarization is typically defined based on textual information alone,
without incorporating insights about a particular decision. As a result, to support risk
analysis of pancreatic cancer, summaries of medical notes may include irrelevant
information such as a knee injury. We propose a novel problem, decision-focused
summarization, where the goal is to summarize relevant information for a decision. We
leverage a predictive model that makes the decision based on the full text to provide
valuable insights on how a decision can be inferred from text. To build a summary, we
then select representative sentences that lead to similar model decisions as using the
full text while accounting for textual non-redundancy. To evaluate our method (Dec
Sum), we build a test bed where the task is to summarize the first ten reviews of a
restaurant in support of predicting its future rating on Yelp. Dec Sum substantially
outperforms text-only summarization methods and model-based explanation methods
in decision faithfulness and representativeness. We further demonstrate that Dec Sum
is the only method that enables humans to outperform random chance in predicting
which restaurant will be better rated in the future.

8. Hierarchical Summarization for Long form Spoken Dialog

Every day we are surrounded by spoken dialog. This medium delivers rich diverse
streams of information auditorily however, systematically understanding dialog can
often be non-trivial. Despite the pervasiveness of spoken dialog, automated speech
understanding and quality information extraction remains markedly poor, especially
when compared to written prose. Furthermore, compared to understanding text,
auditory communication poses many additional challenges such as speaker disfluencies,
informal prose styles, and lack of structure. These concerns all demonstrate the need for
a distinctly speech tailored interactive system to help users understand and navigate the
spoken language domain. While individual automatic speech recognition (ASR) and text
summarization methods already exist, they are imperfect technologies; neither consider
user purpose and intent nor address spoken language induced complications.
Consequently, we design a two stage ASR and text summarization pipeline and propose
a set of semantic segmentation and merging algorithms to resolve these speech
modeling challenges. Our system enables users to easily browse and navigate content as
well as recover from errors in these underlying technologies. Finally, we present an
evaluation of the system which highlights user preference for hierarchical
summarization as a tool to quickly skim audio and identify content of interest to the
user.
CONSOLIDATION OF WORK IN TABLE FORMAT:
PAPER AUTHOR SOURCE DATE
1 Key Point Analysis Via Contrastive MILAD arxiv-cs.CL 2021-09-30
Learning and Extractive Argument ALSHOMARY et. al.
Summarization
2 Privacy Policy Question Answering Moniba arxiv-cs.CL
Assistant: A Query-Guided Keymanesh; Micha
Extractive Summarization Elsner; Srinivasan 2021-09-29
Approach Parthasarathy;

3 Who Says Like A Style of Vitamin: Seolhwa Lee; Kisu arxiv-cs.CL 2021-09-29
Towards Syntax-Aware Dialogue Yang; Chanjun
Summarization Using Multi-task Park; João Sedoc;
Learning Heuiseok Lim;
4 Investigating Entropy for Alka Khurana;
Extractive Document Vasudha
Summarization Bhatnagar; arxiv-cs.IR 2021-09-22

5 Recursively Summarizing Books JEFF WU et. al. arxiv-cs.CL 2021-09-22

with Human Feedback
6 Email Sum: Abstractive Email Shiyue Zhang; Asli ACL 2021-07-30
Thread Summarization Celikyilmaz;
Jianfeng Gao;
Mohit Bansal;
7 Decision-Focused Summarization Chao-Chun Hsu; arxiv-cs.CL 2021-09-14
Chenhao Tan;

8 Hierarchical Summarization for Daniel Li; Thomas arxiv-cs.CL 2021-08-21

Long form Spoken Dialog Chen; Albert Tung;
Lydia Chilton;