ALGOVERSE

AI RESEARCH
Developed by UC Berkeley and Stanford Graduate Student
Instructors and Researchers

Conduct authentic AI research under expert

mentorship. Develop your own techniques and
publish your results in a research paper.

www.algoverse.us/ai-research
NEURIPS TRACK
MAKE AN IMPACT WITH AI
RESEARCH FOR SOCIAL
GOOD

In alignment with the inaugural high school

track at NeurIPS, the world's most prominent AI
conference, we are excited to launch a new
research initiative focused on AI for social
good. Our program will delve into the ethical
development and application of AI technologies
to tackle societal issues. By incorporating
advanced, vision and language models like
GPT4, our curriculum equips students to
contribute meaningfully to this essential area of
research and complete a research paper.
SUMMER
PROGRAM

IMPROVE STATE-OF-THE-ART PERFORMANCE

OF LARGE LANGUAGE MODELS LIKE GPT-4.

We are embarking on an ambitious goal of advancing the frontiers

of large language model capabilities, rigorously evaluating their
performance against industry-standard benchmarks. Leveraging
open source LLMs such as Meta's Llama 2, our program is uniquely
positioned to contribute to this cutting-edge field of research.

Program
(New) NeurIPS Track: May 12 - Jun 30
Lectures on Sundays 1-2:30 pm PT
Summer A: Jun 1 - Aug 17
Lectures on Saturdays 10-11:30 am PT
Summer B: Jun 23 - Sep 8
Lectures on Sundays 10-11:30 am PT
*Lectures are recorded, so you have the option of following
along asynchronously.
Class Format: Meetings are fully online and held over Zoom.
Weekly Structure: The program has 2-3 weekly meetings. We expect
students to dedicate 5-10 hours per week in total, with flexibility for
further exploration.
Weekend Instructor Lecture (1.5 hours): Learn LLM and AI fundamentals and
review relevant literature for research inspiration.
Mid-week Progress Updates (20 minutes, scheduled by group): Share your
weekly progress with your mentor and explore research directions.
Due to the upcoming conference deadline, the NeurIPS track has two mid-week
check-ins. Our summer program has one mid-week check-in.
Office Hours and Slack: Receive support from mentors throughout the
week if you run into roadblocks debugging, want to bounce ideas, or
deep dive into technical topics.
Hands-on Mentorship: Work in a close-knit team of 3-4, guided by a
dedicated mentor who collaborates intimately with the team to facilitate
progress and engages individually to enhance learning.
Program Fee: $1725 USD (priced at ~$60 per instructional hour). We
are priced at a fraction of other research programs; unlike many
research programs, we are genuinely committed to accessibility and an
authentic AI research experience.
 FALL 2023 RESEARCH
HIGHLIGHTS

BOOSTING CHAIN-OF-THOUGHT PROMPTING

WITH CONTRASTIVE DECODING
We propose a contrastive decoding method to increase the effect-
size of chain-of-thought prompting by contrasting the generated
logits of an expert model, using chain of thought, with an amateur
model autoregressively. We demonstrate the performance across
models of varying parameter sizes and amateur model variations.

*Our NAACL submissions are not displayed, as they are in the anonymity period. The works displayed here are still in progress and will be submitted to another *ACL conference. Authors listed in alphabetical order.
 FALL 2023 RESEARCH
HIGHLIGHTS

BENCHMARKING LLM VISUAL-LINGUISTIC

ABILITIES
We propose a novel benchmark to quantify the visual-linguistic
abilities of large language models that emerge with scale. We
contribute a dataset of ASCII text, a learned representation via fine-
tuning CLIP embeddings on our dataset, statistical analysis on its
fidelity, and benchmarks on GPT and Mistral models for ASCII
generation and classification.

*Our NAACL submissions are not displayed, as they are in the anonymity period. The works displayed here are still in progress and will be submitted to another *ACL conference. Authors listed in alphabetical order.
 FALL 2023 RESEARCH
HIGHLIGHTS

OUTLINING ERRORS IN LARGE LANGUAGE

MODELS
Existing prompting methods lack the ability to reflect and correct
errors. We propose error outline prompting (EOP) to recognize errors
across different categories of failure modes and self-correct, serving
as a versatile supplement to conventional CoT.

*Our NAACL submissions are not displayed, as they are in the anonymity period. The works displayed here are still in progress and will be submitted to another *ACL conference. Authors listed in alphabetical order.
 FALL 2023 RESEARCH
HIGHLIGHTS

DISSECTING SIMILARITIES IN SELF-

CONSISTENCY
We evaluate the similarities of generated rationales with BERT
embeddings and propose a few aggregation methods that
outperform naive self consistency on code evaluation, common
sense reasoning, and reasoning tasks. We observe this effect on
models of varying parameter sizes, including GPT3.5, Llama models,
and Mistral models.

*Our NAACL submissions are not displayed, as they are in the anonymity period. The works displayed here are still in progress and will be submitted to another *ACL conference. Authors listed in alphabetical order.

Kevin Zhu
Program Director, Research Mentor
Former UC Berkeley Instructor | Software
Engineer at Palantir | Quant at Citadel
Kevin taught 3000+ Berkeley students during his
tenure as a lecturer for CS198-112 and 5-time Head
GSI, specializing in upper-division algorithms. He
has also taken software engineering roles at Palantir
and various startups, and AI research roles at
Citadel, Goldman Sachs, and Berkeley RISE Lab,
where he applied traditional machine learning
techniques to the stock market and researched
techniques for improving convolutional neural
network inference efficiency. Kevin now serves as
the lead director for the Algoverse programs, as well
as an instructor.
OUR TEAM
Sean O’Brien
AI Research Director
AI Research at UCSD | Former AI
Resident at Meta | Berkeley AI Research
Sean conducts research on large language models
like GPT-4 as a PhD researcher at UCSD. While an
AI resident at Meta, he researched language model
decoding methods and co-authored Shepherd, a
small language model that generates critiques
matching the quality of ChatGPT. Previously, at
Berkeley AI Research (BAIR), he specialized in
transformer architectures for strategy learning. Sean
was also a 7-time GSI at Berkeley, teaching
introductory programming, discrete mathematics, and
upper-division machine learning, while triple majoring
in EECS, math, and cognitive science.
Thomas Lu
Research Mentor
AI Research at CMU | Former AI
Research at Tiktok | Berkeley AI Research
Thomas conducts AI research at Carnegie Mellon
University as a Master's student in machine
learning. He is a co-author of "Learned
Incremental Representations for Parsing", which
earned the highest distinction of Best Paper at
ACL 2022, the premier NLP conference
(reference). He has previously researched at
Berkeley AI Research, MDI, and Tiktok. Thomas
completed his bachelor's at UC Berkeley, triple
majoring in CS, data science, and linguistics with
a 4.0 GPA.
 STUDENT EXPERIENCES

Michael Naeim, Grade 12, Miami College Language High School

"I am thrilled to share my experience with the Algoverse research program. The lectures were exceptionally well-crafted,
providing invaluable insights. The support I received throughout the program was nothing short of amazing. In comparison
to other research programs, Algoverse stands out as I found myself learning a lot from lectures to office hours and weekly
meetings. Embarking on a research project focused on BERT was a daunting task for me and my team, but the
unwavering support from the program made it achievable. The mentors were not just knowledgeable, but also perfect in
their guidance. Their friendliness and constant support made the learning journey truly enjoyable. One aspect that truly
impressed me was the clarity of the plan provided and the abundance of resources at our disposal. The program's
commitment to following up with teams and fostering a sense of community was outstanding. I not only gained valuable
knowledge but also forged connections with like-minded individuals, creating a network of friends who share my interests.
Above all, the mentors were the highlight of the program for me. Their daily support was instrumental in my success. I am
grateful for the experience, the community, and the exceptional guidance I received from the Algoverse research program.
Definitely, I am going to recommend this program to anyone who is both excited to learn about machine learning from
amazing mentors and have a goal to publish his paper at a huge conference like NAACL but also interested in having fun
and friendly experience. I would say mentors and the team you will have is the best part in the program. Support that you
will get from the mentors and how much time they dedicate to helping you is definitely amazing."
 STUDENT EXPERIENCES
"Algoverse was a great experience for me as I learned so much about computer science and AI topics in which I had little
experience with before hand. The mentors Kevin and Sean were both very supportive and knowledgeable in helping the
cohort formulate experiments and develop our research paper. I highly recommend this program to anyone interested in
learning about and being part of the future of computer science."

- Ryan Sarmiento, Grade 12, San Antonio, Texas

"The AI Research program taught me so many valuable skills about large language models, conducting research, and
writing and developing papers that I know will be super useful in college. At the beginning of the semester, I never
thought I had the capabilities to do high level research since it seemed so over my head at the time. But Kevin and Sean
were there to help our group, answer questions at pretty much any hour of the day, and offer expert guidance, advice,
and mentorship that helped us produce tangible results! I’m excited because these are results we can actually submit to
an AI conference in the hopes of getting our work published. This was so much fun, thanks Kevin and Sean!"

- Megan V., Grade 12, Helotes, Texas

"This program was absolutely amazing! I came in not knowing a thing about LLMs and AI, and through the incredibly
helpful lectures and help throughout my research progress, I gained incredibly valuable information and definitely
learned a lot."

- Kerry L., Grade 10, Cary, North Carolina

 STUDENT EXPERIENCES
"The Algoverse AI Research program exceeded my expectations. The intensive, hands-on lectures, world-class instructor
and mentor, and brilliant teammates enabled me to gain invaluable AI research skills I never imagined possible. I believe
my instructors (Sean and Kevin) and teammates were the people I learnt the most from during my 12 weeks. They were
always happy to answer my queries no matter how basic they were. More than the coding expertise or publishing a paper,
the creativity and resilience I built are what I truly cherish. Through late nights coding together, I made many close friends
while realizing my dream of impactful AI contributions. If you are eager to push your abilities and open up new universes
within AI, there's no better place than Algoverse. I can’t wait to see you (yes, you!) pursuing bold innovations in AI at
Algoverse!"
- Ayush C, Grade 12, St. Teresa School

"Algoverse AI Research Program was AWESOME! The flow of the program from learning about the basics of LLM to
critically thinking a real research idea and then implementing it was the most impressive part of the research program.
The level of one-to-one support provided by mentors during the entire program was something I felt special about. It's an
absolute 10/10 experience."

- Yubraj P., Grade 12, Global College International

"As a participant in the Algoverse Research Program, I had a really positive experience. The teaching style was well-
structured, balancing theory and practice... I appreciated the depth of knowledge imparted and the practical skills I
acquired. Overall, my time in the program was a valuable period of learning and growth, helping me to get into the field of
research."
- Tim K., Grade 12, Germany
Contact Us
Email Address
admissions@algoverse.us

Phone Number
Kevin Zhu, Program Director:
630-400-7637

Website
www.algoverse.us/ai-research