Professional Documents
Culture Documents
2023 Emnlp-Main 799
2023 Emnlp-Main 799
2023 Emnlp-Main 799
12931
closer to the goal, Relation: xNeed, Tail: to Symbolic Social Commonsense Knowledge:
take the first step. We use Atomic10x (West • Head: PersonX moves a step closer to the goal
et al., 2022) as our knowledge graph: it includes • Relation: xNeed
• Tail: to take the first step
diverse social (e.g., intention, desire, reaction) and Sentence-form Commonsense: Madeleine took the first
event-centered (e.g., order of events) commonsense. step. Madeleine moves a step closer to the goal.
Since we are interested in distilling social inter- Narrative:
actions, we only retrieve triples related to social Madeleine took the first step towards her goal, and with her
(rather than, e.g., physical) commonsense.3 coach’s encouraging words, she moves one step closer.
Speakers: Madeleine, Coach
2.3 Commonsense Knowledge → Narrative Conversation:
Madeleine: Hey coach, I wanted to talk to you about my
Triple Form to Sentence Form Since common- performance today. I was really pushing myself and I think
sense knowledge graphs are represented in sym- I did pretty well. But I’m still not quite where I want to be.
bolic form (i.e., triples), we first convert them into Coach: Well Madeleine, you’re progressing nicely. You’ve
come a long way since we first started working together. But
simple sentences with templates for each relation. if you want to reach your full potential, there’s still some
For example, the commonsense knowledge in Ta- work to be done.
ble 1 is converted to “Madeleine took the first step. Madeleine: I know that. And I’m willing to put in the
work. It’s just that sometimes I feel like I’m not making as
Madeleine moves a step closer to the goal.” To much progress as I should be. Maybe I’m not training hard
make the sentences sound more natural, we replace enough? Or maybe my technique is off?
Coach: It could be a number of things, Madeleine. But don’t
the person variables (e.g., PersonX, PersonY) with worry, we’ll figure it out together. Let’s just keep working
Top-1K common names of US SSN applicants rang- hard and see how things go.
ing from 1990 to 2021.4 Madeleine: Alright, coach. Thanks for the talk.
Coach: No problem. See you at practice tomorrow.
Sentence Form to Narrative Next, we prompt
Table 1: A sample from S ODA. More examples can
GPT-3.5 to instantiate the sentence-form into a two- be found in Appendix B.
or three-sentence short narrative, e.g., for the com-
monsense example above “ Madeleine took the first 3 S ODA:
step towards her goal, and with her coach’s encour- A Million-scale Social Dialogue Dataset
aging words, she moves one step closer. ” Prior
work has shown that LLMs can effectively gener- We obtain S ODA (SOcial DiAlogues), a large-scale
ate stories with plausible details that go beyond the high-quality conversation dataset covering a wide
contents of the prompt (Radford et al., 2019). range of social interactions, by applying a series
of post-processing (§3.1) to the conversations gen-
2.4 Narrative → Conversation erated from our contextualization framework (§2).
Inferring Conversation Participants Inferring We compare S ODA with existing human-curated di-
the conversation participants from the narrative is alogue corpora (§3.2) and analyze the effectiveness
straightforward in cases where triples contain two of contextualization (§3.3). Table 1 shows a sample
person variables (i.e., PersonX and PersonY). But from our dataset. More details are in Appendix B.
for triples that include only one person (e.g., the
3.1 Post-processing the Conversations
example in Table 1), we query GPT-3.5 to predict
the other interlocutor (e.g., mom, coworker). Basic Filtering Starting with an initial set of 2.2
million conversations sampled from GPT-3.5, we:
Generating Conversation grounded in Narrative (1) use lexical pattern matching to filter out con-
With the narrative and speakers as input, we prompt versations with erroneous patterns – e.g., repetition
GPT-3.5 to generate a full, multi-turn conversation and omission of speaker prefixes (6.3%); (2) re-
between the speakers in the context of the narrative. move conversations that have less than four turns
We append the first speaker as an utterance prefix or more than twenty turns (5.7%); (3) remove con-
to the prompt. Indicating the speakers with prefixes versations with more than two speakers (11.3%);5
helps GPT-3.5 generate fluent conversations that and (4) remove conversations where at least one
alternate between the two. of the speakers was identified as non-human (e.g.,
3
We leave relations for physical and event-centered com- broomstick, imaginary friend, dog; 5.6%).
monsense to potential future work.
4 5
catalog.data.gov/dataset/baby-names-from- Although our pipeline naturally generates multi-party con-
social-security-card-applications-national-data versations as well, we focus on dyadic dialogues in this work.
12932
300
SODA DailyDialog SODA BlendedSkillTalk
250
200
150
100
50
0
Figure 2: Results of head-to-head comparison between dialogues from S ODA, DailyDialog (Li et al., 2017), and
BlendedSkillTalk (Smith et al., 2020) via human judgments (§3.2). The y-axis represents the number of samples
preferred by human judges. The differences in all of the categories except for the Context Dependence comparing
S ODA and BlendedSkillTalk are statistically significant (|z| > 3.3, p < 0.05).
Safety Filtering In order to avoid conversations on the human-annotated subset, with a precision
with dangerous and harmful contents, we apply of 97 for answering “yes". We find 95% of the
two safety filters: Canary (Kim et al., 2022a) and filtered conversations are identified by GPT-3.5 as
Rewire API.6 Canary is a narrative dialogue safety containing the head event. Pairs that lack the head
model that can classify whether the given context event are removed to ensure relevance between
needs caution or intervention. We discard all con- the narrative-conversation pairs and commonsense
versations marked as needing intervention (usually triples. More details are in Appendix B.1.
critical situations, e.g., crimes, emergencies; 4.3%);
Rewire API is a web-based API for detecting toxic Final Dataset After all filtering, 68.9% of the
content. We discard all conversations that are above initial conversations remain, which form the
the threshold of 0.5 for any of the ‘violence’, ‘hate’, 1,486,896 conversations in S ODA.
and ‘sexually explicit’ criteria (∼1%).
Name Bias Mitigation We aim to minimize bi-
Commonsense Filtering We conduct a small- ases associated with specific names while increas-
scale human evaluation via Amazon Mechani- ing inclusion and diversity. Both language mod-
cal Turk with 100 randomly sampled narrative- els and curated datasets often exhibit demographic
conversation pairs (3 annotators per instance) to imbalances (Dinan et al., 2020; Weidinger et al.,
check whether or not the seed commonsense triple 2021; Sheng et al., 2021). Inspired by Smith and
is meaningfully instantiated by the narrative and Williams (2021), we randomly replace all names
conversation. According to majority vote, 88% of in conversations with Top-10K names of US SSN
the instances include the seed commonsense knowl- applicants from 1990 to 2021.7 This covers 95% of
edge. Given that the majority of human-annotated all applicants’ names from the chosen time range
samples include the seed commonsense, we focus window, including various names from diverse gen-
our filtering on excluding narrative-conversation der8 and ethnic backgrounds.
pairs that lack the head event, as they are irrelevant
to the given seed commonsense. 3.2 Comparing S ODA with
To apply this filter to all entries of the corpus, we Human-authored Dialogues
use GPT-3.5 as a zero-shot classifier. As GPT-3.5 High Quality To assess relative quality of the
demonstrated great performance in question an- corpus, we conduct head-to-head human evalu-
swering (Ouyang et al., 2022), we validate the gen- ations on Amazon Mechanical Turk, comparing
erated narrative-conversation pairs by asking the S ODA with two widely used open-domain dialogue
language model itself to judge whether or not the datasets: DailyDialog (Li et al., 2017) and Blended-
head of the commonsense triple is implied. We for- SkillTalk (Smith et al., 2020). We random sample
mulate this as three-way multiple choice questions 300 dialogues from each dataset and evaluate them
(i.e., yes, no, and unknown) and rank the answers according to six criteria (Mehri et al., 2022): (1)
according to their perplexity scores from GPT-3.5.
7
This zero-shot classifier achieves high performance We use Top-1K names when contextualizing the common-
sense triples in §2.3.
6 8
https://rewire.online/ Gender-neutral and nonbinary names are also included.
12933
Avg. Avg. Utt. Lexical Common keywords across all relations
#Dialog
#Turns Length Diversity
friendship, help, support, communication, family,
DailyDialog 13K 7.9 14.6 63.0 car, happiness, school, success, work
PersonaChat 11K 14.8 14.2 43.6
WizardOfWikipedia 22K 9.1 16.4 60.3 Common keywords for each relation (excluding the above)
EmpatheticDialogue 25K 4.3 13.7 64.2 xAttr kindness, anger, intelligent, responsibility, friend,
BlendedSkillTalk 7K 11.2 13.6 64.2 (18%) trust, conversation, food, generosity, smart
ProsocialDialog 58K 5.7 20.0 60.2 xEffect gratitude, anger, upset, hard work, happy, money,
(17%) friend, boss, party, kindness
S ODA 1.5M 7.6 16.1 68.0
xIntent independence, hard work, determination, money,
(23%) relaxation, anger, kindness, store, understanding
Table 2: Statistics of S ODA compared to other large-
scale dialogue datasets. Utt. denotes utterance. Lexical xNeed job, money, confidence, comfort, advice,
diversity is measured with MTLD (McCarthy and Jarvis, (7%) interest, conversation, listening, store, park
2010). Description for each dataset is in Appendix F. xReact frustration, anger, confidence, happy, pride, relief,
(25%) disappointment, relaxation, anxiety, satisfaction
xWant conversation, store, determination, apology, learning,
natural flow, (2) context dependence, (3) topic con- (11%) doctor, job, friend, improvement, marriage
sistency, (4) speaker consistency, (5) specificity,
and (6) overall. Judges are asked to select a better Table 3: Common topic keywords of the narratives (i.e.,
dialogue between the two, regarding each crite- conversation context) in S ODA. Numbers in parentheses
rion. For context dependence, we ask the judges denote the ratio of the relations in S ODA.
to choose which conversation includes responses
that are more dependent on previous turns. Further We find a broad spectrum of topics encountered in
details are in Appendix B.2. social interactions are included in S ODA.
Despite being fully machine-generated, human As a result, conversations in S ODA contain di-
raters judge S ODA as better in quality compared verse lexicons. We compute MTLD (McCarthy and
to both DailyDialog and BlendedSkillTalk across Jarvis, 2010) to measure the lexical diversity of con-
all axes by a large margin, except for the context versations. Table 2 reports the averaged diversity
dependence comparing with BlendedSkillTalk (see of dialogues for each training set. As PersonaChat
Figure 2). In particular, evaluators rate the flow of (Zhang et al., 2018) contains conversations based
S ODA to be significantly more natural than other on a few persona-related sentences, it shows the
human-authored artificial conversation datasets.9 lowest lexical diversity. S ODA, on the other hand,
includes conversations from a variety of social situ-
Large Scale With 1.5 million conversations, ations, which leads to a wider range of words.
S ODA is the largest in scale compared to exist-
Rich Emotion-related Information Since com-
ing crowdsourced open-domain dialogue datasets
monsense knowledge from Atomic10x includes
and the machine-human generated ProsocialDialog
emotional reactions of people to events (i.e., the
dataset (Table 2). It contains more than 11 million
xReact triples), conversations with rich emotional
utterances and each conversation is grounded in
contents are also included in S ODA. In total, S ODA
a short narrative describing the context. In total,
includes 385K conversations generated from 1.7K
S ODA consists of 300 million tokens, making it a
unique emotion descriptions of the xReact triples’
rich source for training conversation models.
Tail (e.g., happy, ashamed, motivated, irritated).11
Diverse Content S ODA is built on top of 1.5 mil- Therefore, it contains significantly more descriptive
lion commonsense knowledge triples of Atomic10x , emotion labels (i.e., the Tail) than other datasets
which have been identified as being softly unique which have fixed number of classes (Li et al., 2017;
(West et al., 2022). Each seed triple is converted Rashkin et al., 2019). Furthermore, because we
to a social narrative that serves as the distinct topic construct conversations in a bottom-up fashion
for each conversation. The Top-10 common key- from those emotion reaction in the commonsense
words from these narratives are listed in Table 3.10 triples, we know which speaker in the conversation
is experiencing the emotion (i.e., PersonX) and
9
A power analysis suggests that with our setup, we can de- what caused the emotion (i.e., the Head event).
tect effect sizes as small as 0.17 with a power and significance
level of 95% (Faul et al., 2014). 11
We note that conversations from other relations also natu-
10
We prompt ChatGPT to output keywords of the narrative. rally include emotional utterances.
12934
DailyDialog BlendedSkillTalk S ODA 100
SODA Conversations Sampled without Context
80
Emotion Ratio Emotion Ratio Emotion Ratio
60
admiration 20.42 curiosity 17.86 curiosity 12.92
gratitude 18.84 admiration 13.16 admiration 11.23 40
curiosity 12.85 sadness 8.50 approval 10.24 20
approval 10.91 joy 5.32 gratitude 7.39 0
joy 4.74 excitement 4.42 joy 6.38 l
Flo
w nc
e cy cy cit
y ess ral
excitement 3.61 surprise 4.34 disappointed 5.41 de ten ten ifi gn Ov
e
u ral p en n sis n sis p ec stin
t e o o S er e
surprise 3.25 disappointed 4.34 confusion 4.68 Na xt
D C rC Int
nte pic ke
love 3.06 fear 4.31 surprise 4.40 o To p ea
C S
optimism 2.94 approval 4.19 realization 3.90
caring 2.23 optimism 3.95 caring 3.77
Figure 3: Results of head-to-head comparison human
evaluation between conversations from S ODA and those
Table 4: The ratio (%) of Top-10 emotions in 10K utter- sampled from GPT-3.5 without context (§3.3). The y-
ances from DailyDialog, BlendedSkillTalk, and S ODA, axis indicates the number of samples that human judges
labeled by the GoEmotions’ 27-emotion-type classifier preferred. The differences are all statistically significant
(Demszky et al., 2020). Full table is in Appendix B.2. with |z| > 2.6, p < 0.05 except for the Natural Flow
class with z = 1.1 and p > 0.05.
We also find the distribution of emotions to be (MTLD; McCarthy and Jarvis, 2010): 68.0 vs 63.1.
less skewed towards specific emotions. To compare
the emotional composition, we use the 27-emotion- 4 C OSMO:
type classifier from GoEmotions (Demszky et al., A Socially Situated Conversation Model
2020) for labeling and compare 10K utterances
from DailyDialog, BlendedSkillTalk, and S ODA. We use S ODA to train C OSMO: a COnverSation
The distribution of emotions for each dataset is pre- MOdel that can converse in a wide range of social
sented in Table 4. S ODA exhibits a more balanced situations. C OSMO can take in situation narrative,
distribution of emotions while maintaining similar along with dialogue history, and generate a next
rankings with other human-authored dialogues. utterance according to a given role.
Cost & Time-Efficient Compared to dialogue Training C OSMO We use several structured
crowdsourcing, collecting S ODA via our contex- components of S ODA during training: (1) the
tualization framework is significantly more time contextual narrative n (§2.3), (2) the perspec-
and cost efficient. With GPT-3.5 text-davinci-002, tive/speaker instruction i (e.g., “Imagine you are
to go from a commonsense triple to a dialogue Madeleine and speak to her coach”) built with the
costs about $0.02, and 10 queries take less than 2 inferred conversation participants (§2.4), and (3)
minutes, counting our full filtration pipeline. the dialogue context c. The model is trained to
generate a target response r when given n, i, and
3.3 Do We Need Contextualization? c – i.e., p(r|n, i, c). We do so in a sequence-to-
To isolate the effect of contextualization (vs. sequence fashion, concatenating n, i, c with a sep-
straightforward sampling from a large language arator <SEP> to serve as input. c is made up of the
model), we compare S ODA with dialogues naively previous conversation utterances concatenated with
sampled from GPT-3.5 without any given context. a turn indicator <TURN>.
We sample 100 dialogues using the same hyperpa- Because conversational models often agree to
rameters and the basic filtering steps in CO 3 , but toxic or unethical behavior (Baheti et al., 2021),
with the following prompt: “The following is for additional training data, we include Prosocial-
a long in-depth conversation between two Dialog (Kim et al., 2022a) (adapted to the same
people.\nPerson 1:.” We ask human judges to format as S ODA, see Appendix C). ProsocialDia-
evaluate the conversations in a head-to-head com- log includes a wide range of negative constructive
parison as before (§3.2), with the additional crite- feedback based on social rules-of-thumb, e.g., “So
rion of interestingness (See et al., 2019). I think it’s best to continue being honest, and apol-
Figure 3 shows that judges significantly prefer ogize that you were lying.” The inclusion of this
context-grounded conversations. Conversations corpus assists conversation models in handling sen-
sampled without context are not only less specific sitive contexts (e.g., biased, harmful, unethical)
and less interesting, but also exhibit lower lexi- without affecting the model performance on other
cal diversity than those from our CO 3 framework datasets (Kim et al., 2022a).
12935
We build C OSMO on top of the LM-adapted T5 Model Natural Consistent Specific Overall
(Raffel et al., 2020; Lester et al., 2021), which BlenderBot-3B 23% 26% 39% 28%
achieves strong benchmark performance across var- C OSMO-3B 77% 74% 61% 72%
ious classification and generation tasks. (Sanh et al., GODELL 13% 14% 15% 14%
2021; Chung et al., 2022). We train two versions C OSMO-3B 87% 86% 85% 86%
of the model: C OSMO -3B and C OSMO -11B using Koala-7B 30% 34% 30% 29%
the T5X library (Roberts et al., 2022). For better C OSMO-3B 70% 66% 70% 71%
robustness and generalizablity to datasets that don’t Vicuna-7B 42% 42% 44% 42%
C OSMO-3B 58% 58% 56% 58%
have contexts or dialogue starting prompts, we ran-
domly drop narrative n and role instruction i 30% Ground Truth 43% 45% 46% 45%
and 50% of the time, respectively. C OSMO-3B 57% 55% 54% 55%
12938
4862, Online and Punta Cana, Dominican Republic. Emily Dinan, Angela Fan, Adina Williams, Jack Ur-
Association for Computational Linguistics. banek, Douwe Kiela, and Jason Weston. 2020.
Queens are powerful too: Mitigating gender bias in
Roy F. Baumeister and Brad J. Bushman. 2017. Social dialogue generation. In Proceedings of the 2020 Con-
Psychology and Human Nature, 4th edition. Cengage ference on Empirical Methods in Natural Language
Learning. Processing (EMNLP), pages 8173–8188, Online. As-
sociation for Computational Linguistics.
Jason Baumgartner, Savvas Zannettou, Brian Keegan,
Megan Squire, and Jeremy Blackburn. 2020. The Emily Dinan, Stephen Roller, Kurt Shuster, Angela
Pushshift Reddit Dataset. In ICWSM. Fan, Michael Auli, and Jason Weston. 2018. Wizard
of Wikipedia: Knowledge-Powered Conversational
Derek Chen and Zhou Yu. 2021. GOLD: Improving Agents. In International Conference on Learning
out-of-scope detection in dialogues using data aug- Representations.
mentation. In Proceedings of the 2021 Conference on
Empirical Methods in Natural Language Processing, F Faul, E Erdfelder, AG Lang, and A Buchner. 2014.
pages 429–442, Online and Punta Cana, Dominican G* power: statistical power analyses for windows
Republic. Association for Computational Linguistics. and mac.
Xinyang Geng, Arnav Gudibande, Hao Liu, Eric Wal-
Maximillian Chen, Alexandros Papangelis, Chenyang lace, Pieter Abbeel, Sergey Levine, and Dawn Song.
Tao, Seokhwan Kim, Andy Rosenbaum, Yang Liu, 2023. Koala: A dialogue model for academic re-
Zhou Yu, and Dilek Hakkani-Tur. 2023. PLACES: search. Blog post.
Prompting language models for social conversation
synthesis. In Findings of the Association for Com- Fritz Heider. 1958. The Psychology of Interpersonal
putational Linguistics: EACL 2023, pages 844–868, Relations. Psychology Press.
Dubrovnik, Croatia. Association for Computational
Linguistics. Ari Holtzman, Peter West, Vered Shwartz, Yejin Choi,
and Luke Zettlemoyer. 2021. Surface form com-
Maximillian Chen, Alexandros Papangelis, Chenyang petition: Why the highest probability answer isn’t
Tao, Andy Rosenbaum, Seokhwan Kim, Yang Liu, always right. In Proceedings of the 2021 Conference
Zhou Yu, and Dilek Hakkani-Tur. 2022. Weakly on Empirical Methods in Natural Language Process-
supervised data augmentation through prompting for ing, pages 7038–7051, Online and Punta Cana, Do-
dialogue understanding. NeurIPS 2022 Workshop minican Republic. Association for Computational
SyntheticData4ML. Linguistics.
Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Jena D Hwang, Chandra Bhagavatula, Ronan Le Bras,
Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Jeff Da, Keisuke Sakaguchi, Antoine Bosselut, and
Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Yejin Choi. 2021. (comet-) atomic 2020: On sym-
Stoica, and Eric P. Xing. 2023. Vicuna: An open- bolic and neural commonsense knowledge graphs.
source chatbot impressing gpt-4 with 90%* chatgpt In Proceedings of the AAAI Conference on Artificial
quality. Intelligence, volume 35, pages 6384–6392.
Katharina Kann, Abteen Ebrahimi, Joewie Koh, Shiran
Hyung Won Chung, Le Hou, Shayne Longpre, Bar- Dudy, and Alessandro Roncone. 2022. Open-domain
ret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi dialogue generation: What we can do, cannot do,
Wang, Mostafa Dehghani, Siddhartha Brahma, et al. and should do next. In Proceedings of the 4th Work-
2022. Scaling instruction-finetuned language models. shop on NLP for Conversational AI, pages 148–165,
arXiv preprint arXiv:2210.11416. Dublin, Ireland. Association for Computational Lin-
guistics.
Kate Crawford. 2021. Atlas of AI. Yale University
Press. Hyunwoo Kim, Youngjae Yu, Liwei Jiang, Ximing
Lu, Daniel Khashabi, Gunhee Kim, Yejin Choi, and
Cristian Danescu-Niculescu-Mizil and Lillian Lee. 2011. Maarten Sap. 2022a. ProsocialDialog: A prosocial
Chameleons in imagined conversations: A new ap- backbone for conversational agents. In Proceedings
proach to understanding coordination of linguistic of the 2022 Conference on Empirical Methods in Nat-
style in dialogs. In Proceedings of the 2nd Workshop ural Language Processing, pages 4005–4029, Abu
on Cognitive Modeling and Computational Linguis- Dhabi, United Arab Emirates. Association for Com-
tics, pages 76–87, Portland, Oregon, USA. Associa- putational Linguistics.
tion for Computational Linguistics.
Minju Kim, Chaehyeong Kim, Yong Ho Song, Seung-
Dorottya Demszky, Dana Movshovitz-Attias, Jeongwoo won Hwang, and Jinyoung Yeo. 2022b. BotsTalk:
Ko, Alan Cowen, Gaurav Nemade, and Sujith Ravi. Machine-sourced framework for automatic curation
2020. GoEmotions: A dataset of fine-grained emo- of large-scale multi-skill dialogue datasets. In Pro-
tions. In Proceedings of the 58th Annual Meeting of ceedings of the 2022 Conference on Empirical Meth-
the Association for Computational Linguistics, pages ods in Natural Language Processing, pages 5149–
4040–4054, Online. Association for Computational 5170, Abu Dhabi, United Arab Emirates. Association
Linguistics. for Computational Linguistics.
12939
Jonáš Kulhánek, Vojtěch Hudeček, Tomáš Nekvinda, in Natural Language Processing, pages 1635–1648,
and Ondřej Dušek. 2021. AuGPT: Auxiliary tasks Abu Dhabi, United Arab Emirates. Association for
and data augmentation for end-to-end dialogue with Computational Linguistics.
pre-trained language models. In Proceedings of the
3rd Workshop on Natural Language Processing for Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida,
Conversational AI, pages 198–210, Online. Associa- Carroll L Wainwright, Pamela Mishkin, Chong
tion for Computational Linguistics. Zhang, Sandhini Agarwal, Katarina Slama, Alex
Ray, John Schulman, Jacob Hilton, Fraser Kelton,
Brian Lester, Rami Al-Rfou, and Noah Constant. 2021. Luke Miller, Maddie Simens, Amanda Askell, Pe-
The power of scale for parameter-efficient prompt ter Welinder, Paul Christiano, Jan Leike, and Ryan
tuning. In Proceedings of the 2021 Conference on Lowe. 2022. Training Language Models to Follow
Empirical Methods in Natural Language Processing, Instructions with Human Feedback. arXiv preprint
pages 3045–3059, Online and Punta Cana, Domini- arXiv:2203.02155.
can Republic. Association for Computational Lin-
guistics. Baolin Peng, Michel Galley, Pengcheng He, Chris
Brockett, Lars Liden, Elnaz Nouri, Zhou Yu, Bill
Yanran Li, Hui Su, Xiaoyu Shen, Wenjie Li, Ziqiang Dolan, and Jianfeng Gao. 2022. GODEL: large-scale
Cao, and Shuzi Niu. 2017. DailyDialog: A manually pre-training for goal-directed dialog. arXiv preprint
labelled multi-turn dialogue dataset. In Proceedings arXiv:2206.11309.
of the Eighth International Joint Conference on Nat-
ural Language Processing (Volume 1: Long Papers), Alec Radford, Jeffrey Wu, Rewon Child, David Luan,
pages 986–995, Taipei, Taiwan. Asian Federation of Dario Amodei, Ilya Sutskever, et al. 2019. Language
Natural Language Processing. Models are Unsupervised Multitask Learners. Ope-
nAI blog, 1(8):9.
Zekun Li, Wenhu Chen, Shiyang Li, Hong Wang, Jing
Qian, and Xifeng Yan. 2022. Controllable dialogue Colin Raffel, Noam Shazeer, Adam Roberts, Katherine
simulation with in-context learning. In Findings Lee, Sharan Narang, Michael Matena, Yanqi Zhou,
of the Association for Computational Linguistics: Wei Li, and Peter J Liu. 2020. Exploring the Lim-
EMNLP 2022, pages 4330–4347, Abu Dhabi, United its of Transfer Learning with a Unified Text-to-Text
Arab Emirates. Association for Computational Lin- Transformer. Journal of Machine Learning Research,
guistics. 21:1–67.
Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Hannah Rashkin, Antoine Bosselut, Maarten Sap, Kevin
Ruochen Xu, and Chenguang Zhu. 2023. GPTeval: Knight, and Yejin Choi. 2018. Modeling naive psy-
NLG Evaluation using GPT-4 with Better Human chology of characters in simple commonsense stories.
Alignment. arXiv preprint arXiv:2303.16634. In Proceedings of the 56th Annual Meeting of the As-
sociation for Computational Linguistics (Volume 1:
Raymond A Mar and Keith Oatley. 2008. The function Long Papers), pages 2289–2299, Melbourne, Aus-
of fiction is the abstraction and simulation of social tralia. Association for Computational Linguistics.
experience. Perspectives on psychological science,
3(3):173–192. Hannah Rashkin, Eric Michael Smith, Margaret Li, and
Y-Lan Boureau. 2019. Towards empathetic open-
Philip M McCarthy and Scott Jarvis. 2010. Mtld, vocd-
domain conversation models: A new benchmark and
d, and hd-d: A validation study of sophisticated ap-
dataset. In Proceedings of the 57th Annual Meet-
proaches to lexical diversity assessment. Behavior
ing of the Association for Computational Linguistics,
research methods, 42(2):381–392.
pages 5370–5381, Florence, Italy. Association for
Shikib Mehri, Jinho Choi, Luis Fernando D’Haro, Jan Computational Linguistics.
Deriu, Maxine Eskenazi, Milica Gasic, Kallirroi
Georgila, Dilek Hakkani-Tur, Zekang Li, Verena Rob Reich, Mehran Sahami, and Jeremy M Weinstein.
Rieser, Samira Shaikh, David Traum, Yi-Ting Yeh, 2021. System error: Where big tech went wrong and
Zhou Yu, Yizhe Zhang, and Chen Zhang. 2022. Re- how we can reboot. Hodder & Stoughton.
port from the nsf future directions workshop on auto-
Alan Ritter, Colin Cherry, and William B. Dolan. 2011.
matic evaluation of dialog: Research directions and
Data-driven response generation in social media. In
challenges. arXiv preprint arXiv:2203.10012.
Proceedings of the 2011 Conference on Empirical
Rauni Myllyniemi. 1986. Conversation as a system of Methods in Natural Language Processing, pages 583–
social interaction. Language & Communication. 593, Edinburgh, Scotland, UK. Association for Com-
putational Linguistics.
OpenAI. 2022. Chatgpt: Optimizing language models
for dialogue. Adam Roberts, Hyung Won Chung, Anselm Levskaya,
Gaurav Mishra, James Bradbury, Daniel Andor, Sha-
Jiao Ou, Jinchao Zhang, Yang Feng, and Jie Zhou. 2022. ran Narang, Brian Lester, Colin Gaffney, Afroz
Counterfactual data augmentation via perspective Mohiuddin, Curtis Hawthorne, Aitor Lewkowycz,
transition for open-domain dialogues. In Proceed- Alex Salcianu, Marc van Zee, Jacob Austin, Se-
ings of the 2022 Conference on Empirical Methods bastian Goodman, Livio Baldini Soares, Haitang
12940
Hu, Sasha Tsvyashchenko, Aakanksha Chowdh- Eric Smith, Orion Hsu, Rebecca Qian, Stephen Roller,
ery, Jasmijn Bastings, Jannis Bulian, Xavier Gar- Y-Lan Boureau, and Jason Weston. 2022. Human
cia, Jianmo Ni, Andrew Chen, Kathleen Kenealy, evaluation of conversations is an open problem: com-
Jonathan H. Clark, Stephan Lee, Dan Garrette, James paring the sensitivity of various methods for eval-
Lee-Thorp, Colin Raffel, Noam Shazeer, Marvin uating dialogue agents. In Proceedings of the 4th
Ritter, Maarten Bosma, Alexandre Passos, Jeremy Workshop on NLP for Conversational AI, pages 77–
Maitin-Shepard, Noah Fiedel, Mark Omernick, Bren- 97, Dublin, Ireland. Association for Computational
nan Saeta, Ryan Sepassi, Alexander Spiridonov, Linguistics.
Joshua Newlan, and Andrea Gesmundo. 2022. Scal-
ing up models and data with t5x and seqio. arXiv Eric Michael Smith and Adina Williams. 2021. Hi,
preprint arXiv:2203.17189. my name is martha: Using names to measure and
mitigate bias in generative dialogue models. arXiv
Stephen Roller, Emily Dinan, Naman Goyal, Da Ju, preprint arXiv:2109.03300.
Mary Williamson, Yinhan Liu, Jing Xu, Myle Ott, Eric Michael Smith, Mary Williamson, Kurt Shuster,
Eric Michael Smith, Y-Lan Boureau, and Jason We- Jason Weston, and Y-Lan Boureau. 2020. Can you
ston. 2021. Recipes for building an open-domain put it all together: Evaluating conversational agents’
chatbot. In Proceedings of the 16th Conference of ability to blend skills. In Proceedings of the 58th
the European Chapter of the Association for Compu- Annual Meeting of the Association for Computational
tational Linguistics: Main Volume, pages 300–325, Linguistics, pages 2021–2030, Online. Association
Online. Association for Computational Linguistics. for Computational Linguistics.
David E Rumelhart. 1975. Notes on a schema for stories. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier
In Representation and understanding, pages 211–236. Martinet, Marie-Anne Lachaux, Timothée Lacroix,
Elsevier. Baptiste Rozière, Naman Goyal, Eric Hambro,
Faisal Azhar, et al. 2023. Llama: Open and effi-
Victor Sanh, Albert Webson, Colin Raffel, Stephen cient foundation language models. arXiv preprint
Bach, Lintang Sutawika, Zaid Alyafeai, Antoine arXiv:2302.13971.
Chaffin, Arnaud Stiegler, Arun Raja, Manan Dey,
et al. 2021. Multitask prompted training enables Nhat Tran, Malihe Alikhani, and Diane Litman. 2022.
zero-shot task generalization. In International Con- How to ask for donations? learning user-specific
ference on Learning Representations. persuasive dialogue policies through online interac-
tions. In Proceedings of the 30th ACM Conference
on User Modeling, Adaptation and Personalization,
Maarten Sap, Hannah Rashkin, Derek Chen, Ronan
pages 12–22.
Le Bras, and Yejin Choi. 2019. Social IQa: Com-
monsense reasoning about social interactions. In Ben Wang. 2021. Mesh-Transformer-JAX: Model-
Proceedings of the 2019 Conference on Empirical Parallel Implementation of Transformer Lan-
Methods in Natural Language Processing and the guage Model with JAX. https://github.com/
9th International Joint Conference on Natural Lan- kingoflolz/mesh-transformer-jax.
guage Processing (EMNLP-IJCNLP), pages 4463–
4473, Hong Kong, China. Association for Computa- Laura Weidinger, John Mellor, Maribeth Rauh, Conor
tional Linguistics. Griffin, Jonathan Uesato, Po-Sen Huang, Myra
Cheng, Mia Glaese, Borja Balle, Atoosa Kasirzadeh,
Roger C Schank and Robert P Abelson. 1975. Scripts, et al. 2021. Ethical and social risks of harm from
Plans, and Knowledge. In IJCAI. language models. arXiv preprint arXiv:2112.04359.
Peter West, Chandra Bhagavatula, Jack Hessel, Jena
Abigail See, Stephen Roller, Douwe Kiela, and Jason
Hwang, Liwei Jiang, Ronan Le Bras, Ximing Lu,
Weston. 2019. What makes a good conversation?
Sean Welleck, and Yejin Choi. 2022. Symbolic
how controllable attributes affect human judgments.
knowledge distillation: from general language mod-
In Proceedings of the 2019 Conference of the North
els to commonsense models. In Proceedings of the
American Chapter of the Association for Computa-
2022 Conference of the North American Chapter of
tional Linguistics: Human Language Technologies,
the Association for Computational Linguistics: Hu-
Volume 1 (Long and Short Papers), pages 1702–1723,
man Language Technologies, pages 4602–4625, Seat-
Minneapolis, Minnesota. Association for Computa-
tle, United States. Association for Computational
tional Linguistics.
Linguistics.
Noam Shazeer and Mitchell Stern. 2018. Adafactor: Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur
Adaptive learning rates with sublinear memory cost. Szlam, Douwe Kiela, and Jason Weston. 2018. Per-
In International Conference on Machine Learning. sonalizing dialogue agents: I have a dog, do you
PMLR. have pets too? In Proceedings of the 56th Annual
Meeting of the Association for Computational Lin-
Emily Sheng, Josh Arnold, Zhou Yu, Kai-Wei Chang, guistics (Volume 1: Long Papers), pages 2204–2213,
and Nanyun Peng. 2021. Revealing persona biases in Melbourne, Australia. Association for Computational
dialogue systems. arXiv preprint arXiv:2104.08728. Linguistics.
12941
Chujie Zheng, Sahand Sabour, Jiaxin Wen, and Min-
lie Huang. 2023. AugESC: Large-scale data aug-
mentation for emotional support conversation with
pre-trained language models. In Findings of ACL.
Pei Zhou, Hyundong Cho, Pegah Jandaghi, Dong-Ho
Lee, Bill Yuchen Lin, Jay Pujara, and Xiang Ren.
2022. Reflect, not reflex: Inference-based common
ground improves dialogue response quality. In Pro-
ceedings of the 2022 Conference on Empirical Meth-
ods in Natural Language Processing, pages 10450–
10468, Abu Dhabi, United Arab Emirates. Associa-
tion for Computational Linguistics.
Pei Zhou, Karthik Gopalakrishnan, Behnam Hedayat-
nia, Seokhwan Kim, Jay Pujara, Xiang Ren, Yang
Liu, and Dilek Hakkani-Tur. 2021. Commonsense-
focused dialogues for response generation: An em-
pirical study. In Proceedings of the 22nd Annual
Meeting of the Special Interest Group on Discourse
and Dialogue, pages 121–132, Singapore and Online.
Association for Computational Linguistics.
12942
A Details of CO 3 Relation Template for sentence form
xReact [Head]. Now PersonX feels [Tail].
A.1 Commonsense Knowledge → Narrative
xIntent [Head] because PersonX wants [Tail].
Retrieving Social Commonsense Knowledge
xAttr PersonX is [Tail]. [Head].
We use the x-relations from Atomic10x (West et al.,
xEffect [Head]. Now PersonX [Tail].
2022), which are the inferences of people’s mental
states: xIntent, xWant, xReact, xAttr, and xWant [Head]. Now PersonX wants [Tail].
xNeed. Table 3 summarizes the ratio of relations xNeed PersonX [Tail in past tense]. [Head].
included in our S ODA dataset. We leave other rela-
tions (e.g., isBefore, isAfter) for future work. Table 8: Templates for converting symbolic common-
sense knowledge to sentence form.
Triple Form to Sentence Form Table 8 lists the
templates for converting symbolic commonsense Relation Template for building validation questions
knowledge to sentence form.
xReact Does PersonX feel [Tail] after [Head]?
Sentence Form to Narrative We prompt xIntent Does PersonX intend [Tail] when [Head]?
GPT-3.5 with “[sentence-form commonsense] xAttr Can PersonX be considered [Tail] when
Rewrite this story with more specific [Head]?
details in two or three sentences:”. We xEffect [Head]. As a result, PersonX [Tail]. Is this
find long narratives tend to be driven far away from true?
the original commonsense knowledge. Therefore, xWant Does PersonX want [Tail] after [Head]?
we set the length of the narrative to two or three xNeed [Tail in past tense]. Is this true when [Head]?
sentences.
We leverage text-davinci-002 GPT-3.5 for Table 9: Templates for converting symbolic common-
generating narratives. We set temperature to 0.9, sense knowledge to questions for validation.
top-p to 0.95, frequency penalty to 1.0, presence
penalty to 0.6, and max tokens to 1024.
B Details of S ODA
A.2 Narrative → Conversation Table 10 and Table 11 show samples from our
dataset.
Inferring Conversation Participants We
prompt GPT-3.5 with “[narrative] The B.1 Post-processing the Conversations
following is a conversation in the scene
Filtering Non-human Speakers First, we check
between [PersonX’s name] and ...” to let it
whether the speaker prefix includes the name from
finish the partial prompt. This yields a plausible
our name base (§2.4). Next, we use lexical pat-
interlocutor for a given narrative (e.g., mom,
tern matching and identify words in speaker pre-
classmate, coworker, etc.); for the example story
fixes that indicate humans (e.g., mom, dad, teacher,
with Madeleine, “her coach” was predicted.
Mrs., Mr.). Finally, for speaker prefixes that
We leverage the text-davinci-002 GPT-3.5 do not match the above patterns, we prompt the
model for identifying the speakers. We set tem- text-davinci-002 GPT-3.5 model whether the
perature to 0, top-p to 1.0, frequency penalty to 0, speaker is human. For example, “Q: Is [speaker
presence penalty to 0, and max tokens to 16. prefix] a person?\nA:".”
Generating Conversation Grounded in Narra- Filtering with Commonsense Triples Using a
tive We again leverage the text-davinci-002 prompt, we ask two questions about the Head
GPT-3.5 model for generating conversations. An event and also the Relation-Tail event for each
example prompt is “[narrative] The following instance: (1) is the head of the triple repre-
is a long in-depth conversation happening sented in the narrative-conversation pair; and (2)
in the scene between Madeleine and her are the relation and tail? We prompt GPT-3.5
coach with multiple turns.\nMadeleine:”. with “[narrative]\nQ: [head question]\nA:”
We use the same hyperparameter setting as the nar- and “[conversation]\nQ: [relation-tail
rative generation. question]\nA:” Table 9 lists the templates for
12943
Symbolic Social Commonsense Knowledge: Symbolic Social Commonsense Knowledge:
• Head: PersonX provides another service • Head: PersonX takes on a lot of work
• Relation: xIntent • Relation: xReact
• Tail: to be a helpful person • Tail: pressured
Sentence-form Commonsense: Jabriel provides another Sentence-form Commonsense: Yamir takes on a lot of
service because Jabriel wants to be a helpful person. work. Now Yamir feels pressured.
Narrative: Narrative:
Yamir is a high school student who often takes on too much
Jabriel provides a service by taking care of people’s pets
work. She frequently signs up for more extracurricular activ-
while they are out of town. Jabriel is a helpful person who
ities than she can handle and regularly agrees to do favors
loves animals, so she decided to start this business. She has
for her friends, even when she’s already overwhelmed with
been taking care of pets for three years and has never had
other commitments. As a result, Yamir is constantly stressed
any problems.
out and feels like she can never catch up.
Speakers: Jabriel, Client Speakers: Yamir, Lily
Conversation: Conversation:
Jabriel: Hi there! Are you looking for someone to take care Yamir: I can’t believe I agreed to do this. I’m already so
of your pet while you’re out of town? behind on everything else.
Client: Yes, I am. I’m going on vacation for a week and I Lily: What’s wrong? You seem really stressed out.
need someone to watch my cat. Yamir: I am. I feel like I can never catch up. Every time
Jabriel: That’s no problem at all! I love taking care of I turn around, there’s something else that needs to be done.
animals, and I have experience with cats. I’ll be happy to And it’s all because I keep taking on too much.
take care of your cat while you’re gone. Lily: Why do you keep doing that?
Client: That sounds great. How much do you charge? Yamir: I don’t know. I guess I just feel like if I don’t, then
Jabriel: I charge $20 per day, plus an additional $10 per day I’m not really doing my part or contributing enough. But it’s
if you want me to stay overnight. just too much sometimes and it makes me so stressed out.
Client: Sounds reasonable. I’ll definitely need you to stay Lily: Well, maybe you need to start saying no more often.
overnight, so it will be $30 per day. When can you start? Just because someone asks you to do something doesn’t
Jabriel: Any time that works for you! Just let me know mean you have to say yes. You have a right to say no and set
when you’ll be leaving and when you’ll be back, and I’ll your own limits.
take care of everything in between. Yamir: Yeah, maybe you’re right...I should start saying no
more often. It’s just so hard though when people are counting
Client: Alright, I’ll be leaving on Saturday and I’ll be back
on me and asking for favors all the time. But maybe if I start
next Saturday. Can you come over on Friday evening to
setting some boundaries, it’ll help reduce my stress levels in
meet my cat and get everything set up?
the long run. Thanks for talking with me about this, Lily - it
Jabriel: Sounds perfect. I’ll see you on Friday at 6pm. really helped put things into perspective!"
Table 10: A sample from S ODA. Table 11: Another sample from S ODA.
building questions for commonsense validation. 3.5 on 100 human-annotated samples for com-
For example, the commonsense knowledge triple monsense validation. We ask three human judges
in Table 1 will accompany questions of “Madeleine with the same question-answer format given to the
moves a step closer to the goal, is this true?” and model for each triple-narrative-conversation pair.
“Madeleine took the first step. Is this true when
Madeleine moves a step closer to the goal?” We
B.2 Comparing S ODA with
formulate this as a three-way multiple choice ques-
Human-authored Dialogues
tion and rank answers (i.e., yes, no, and unknown)
according to the perplexity score using conditional Figure 4 shows the annotation page for workers
pointwise mutual information (Holtzman et al., evaluating the dialogue quality.
2021). We ask the questions with and without the
context (i.e., the narrative and conversation). Table IRB Information Crowdworking studies of stan-
9 lists the templates for building questions for com- dard NLP corpora (involving no personal disclo-
monsense validation. We find 66%, 95%, and 68% sures) are not required by our IRB to be reviewed
of filtered conversations are identified by GPT-3.5 by them. While the authors of this work are not
as containing the full commonsense triple, the head lawyers and this is not legal advice, this opinion is
event, and the relation-tail event, respectively: in based on United States federal regulation 45 CFR
total, 1,003,595 conversations are identified as fully 46, under which this study qualifies as exempt. We
encapsulating the seed commonsense knowledge. do not release crowdworker IDs, so annotations
Table 13 summarizes the performance of GPT- cannot be back-traced to individual workers.
12944
DailyDialog BlendedSkillTalk S ODA Precision Recall F1-score
Emotion Ratio Emotion Ratio Emotion Ratio
Head
admiration 20.42 curiosity 17.86 curiosity 12.92
gratitude 18.84 admiration 13.16 admiration 11.23 Yes 98.9 94.8 96.8
curiosity 12.85 sadness 8.50 approval 10.24 No 00.0 00.0 00.0
approval 10.91 joy 5.32 gratitude 7.39 Unknown 16.7 100.0 28.6
joy 4.74 excitement 4.42 joy 6.38 Overall 96.1 93.0 94.2
excitement 3.61 surprise 4.34 disappointed 5.41
surprise 3.25 disappointed 4.34 confusion 4.68 Head /w PMI
love 3.06 fear 4.31 surprise 4.40 Yes 96.9 96.9 96.9
optimism 2.94 approval 4.19 realization 3.90 No 00.0 00.0 00.0
caring 2.23 optimism 3.95 caring 3.77
Unknown 00.0 00.0 00.0
remorse 2.07 realization 3.84 sadness 3.76
disapproval 1.95 annoyance 3.48 excitement 3.20
Overall 94.0 94.0 94.0
fear 1.82 love 2.97 remorse 2.81
Relation-Tail
sadness 1.77 confusion 2.54 disapproval 2.74
disappointed 1.47 caring 2.31 annoyance 2.35 Yes 89.2 76.7 82.5
annoyance 1.41 disgust 1.99 desire 2.31 No 21.4 42.9 28.6
confusion 1.23 nervousness 1.88 optimism 2.23 Unknown 8.3 14.3 10.5
realization 1.12 remorse 1.76 love 1.88
Overall 78.8 70.0 73.7
anger 0.97 anger 1.68 fear 1.81
amusement 0.92 embarrassed 1.44 anger 1.75 Relation-Tail /w PMI
desire 0.89 disapproval 1.41 nervousness 1.45
disgust 0.51 amusement 1.09 relief 0.99 Yes 92.2 68.6 78.7
nervousness 0.27 desire 1.09 embarrassed 0.82 No 21.4 42.9 28.6
embarrassed 0.22 pride 0.74 disgust 0.58 Unknown 16.7 85.7 27.9
pride 0.21 gratitude 0.66 pride 0.47 Overall 80.4 65.0 69.6
relief 0.21 relief 0.58 amusement 0.41
grief 0.00 grief 0.00 grief 0.00
Table 13: Evaluation results of commonsense validation
for short question-answering with InstructGPT on 100
Table 12: The ratio (%) of emotions in 10K utterances
human-annotated samples.
from DailyDialog, BlendedSkillTalk, and S ODA, la-
beled by the 27-emotion-type classifier from GoEmo-
tions (Demszky et al., 2020).
Converting ProsocialDialog to S ODA format
We randomly sample names from our name
Analysis on Emotion Distribution To obtain database (§2.3) to construct the situation descrip-
emotional responses, we randomly sample 10K ut- tions and perspective instructions for ProsocialDia-
terances with emotion labels from DailyDialog (Li log. The situation descriptions are made from the
et al., 2017), utterances in conversations with the RoTs in ProsocialDialog (e.g., “Cosmo is trying to
EmpatheticDialogue (Rashkin et al., 2019) theme gently convince a friend it’s wrong to think all men
for BlendedSkillTalk (Smith et al., 2020), and ut- are violent.”); the instructions are built as we did
terances in conversations generated from xReact for S ODA (§4).
triples for S ODA. We run the finetuned BERT-base
classifier (Demszky et al., 2020) on each utterance. D Experiment Details
Table 12 shows the full distribution across 27 emo-
tion types for each dataset. Automatic Evaluation via GPT-4 Inspired
by Liu et al. (2023), we run automatic evalu-
Statistics of Human Evaluation A total of 74 ation on the overall quality of responses with
workers participated in comparing dialogues, yield- GPT-4. We use the same head-to-head com-
ing a Krippendorf’s alpha of 0.25. This indicates parison setup from Table 5 and 6 with the
fair agreements on the quality judgments. following prompt given to GPT-4: “You are
a response evaluator. Your task is
C Details of C OSMO to choose the overall better response
out of the two given the following
Training Details C OSMO -3B/C OSMO -11B are context. You should consider naturalness,
trained using v3-32/v3-128 TPU accelerators specificity, naturalness, and
with batch size 256 (effective batch ≈ 780) consistency.\n\nContext:\n{CONTEXT}\n\n1)
for 110K/130K additional steps using Adafactor {RESPONSE}\n2) {RESPONSE}\n\nQuestion:
(Shazeer and Stern, 2018) with constant learning Which response is better in terms of
rate .001. overall quality?\nAnswer: Response ”.
12945
Dialogue Context: Model Natural Consistent Specific Overall
A: I really need to start eating healthier. BlendedSkillTalk
B: I have to start eating better too. Koala-7B 26% 27% 35% 25%
A: What kind of food do you usually eat? C OSMO-3B 74% 73% 65% 75%
B: I try my best to eat only fruits, vegetables, and chicken.
Vicuna-7B 43% 47% 45% 46%
A: Is that really all that you eat? C OSMO-3B 57% 53% 55% 54%
B: That’s basically it.
A: How do you stick to only those foods? Table 16: Human evaluation results for head-to-head
Ground-truth response:
comparison of model responses under zero-shot setting
with C OSMO, Koala (Geng et al., 2023), and Vicuna
Actually, fruits and veggies are really good for you.
(Chiang et al., 2023). BlendedSkillTalk (Smith et al.,
DialoGPT: I eat a lot of fruit and veggies. I stick to a lot of 2020) is an unseen dataset for all three models.
things. I don’t eat a lot of junk food.
12948
Figure 5: The annotation page for evaluating responses on Amazon Mechanical Turk.
12949