Improve the factuality

of your generative AI
apps by grounding
responses in your data

Proprietary
 Lewis Tom
Liu Beyer
Group Product Manager, Group Product Manager,
Google Cloud Google Cloud

Google Cloud Next ‘24 Proprietary 03

 01 Factuality & Grounding
Agenda 02 Grounding with Google Search
03 RAG on Vertex AI
04 Grounding in your data
05 Check Grounding

Proprietary 04
Factuality &
Grounding

Google Cloud Next ‘24 Proprietary 05

 Grounding is absolutely
essential for Gen AI
applications.

Google Cloud Next ‘24 06

 1. Improve factuality
2. Access up-to-date information
3. Connect an answer to its source

Google Cloud Next ‘24 07

Vertex AI is introducing a
comprehensive Grounding solution
World knowledge Private knowledge

Google Search Your data RAG RAG Validation

Ground your response in Use Grounded Orchestration & Analysis
the world's knowledge Generation API to
Use the RAG API and Use Check Grounding to
with Google Search ground in your own data.
Reasoning Engine to validate grounding of a
grounding for Gemini.
build custom RAG generated response.
workflows

Google Cloud Next ‘24 Proprietary 08

Grounding with
Google Search for
Gemini

Google Cloud Next ‘24 Proprietary 09

LLMs can make
things up and harm
user trust

LLMs may say things that are not factual*

(aka hallucinate), especially regarding
topics it didn’t learn about during training:

- New information created since training

- Niche knowledge
 Natively Sophisticated Advanced
multimodal reasoning Coding
The
1M long context Highly
window efficient
era
is here Now, grounding
with Google
Search

Google Cloud Next ‘24 Proprietary

Announce Google Search Grounding for Gemini
1. Anchor model responses
LLM response directly based on trusted Google Search
world knowledge and public facts

2. Source data provided, reducing

hallucinations
Grounding provides corroboration urls, so that you can
confirm the evidence and identify hallucinations more
easily
3. Working out-of-box without any
development needed
Enable the feature with simple configuration to different
data sources

Google Cloud Next ‘24 Proprietary

Customers building
GenAI apps on Cloud’s
Vertex LLM API

● Build user trust

● Send explainable answers
● Provide a range of Perspectives
● Inspire curiosity

Google Cloud Next ‘24 Proprietary 013

Who wins Oscar for Best Actress this year?

Ungrounded Gemini Grounded Gemini

The 95th Academy Awards, honoring the best

films of 2022, took place on March 12, 2023. Emma Stone won the Oscar for Best Actress in
Here's the winner of the Best Actress category: 2024 for her role in “Poor Things." [1]....
Michelle Yeoh won the award for her role as
Evelyn Wang in Everything Everywhere All at
Once…
Grounding sources
1. Who won Oscars for 2024?, …[↗]

Google Cloud Next ‘24 Proprietary

 Search Grounding

Tap inline source links to open

the source webpage within
Uber Eats’s webview

developing new
experience and
interaction with Google
Search grounding

Suggested Searches

Tap Google Search suggestion to

open the search results page
within Uber Eats’s webview

Proprietary 015
Quora Poe Tap inline source links to
open the source webpage

Platform to developers
to build and share
chatbot. It has millions
of custom bots in one
place, with native apps
for every device.

Tap Google Search suggestion to open the

search results page within Quora’s webview

Google Cloud Next ‘24 Proprietary 016

RAG on Vertex AI

Google Cloud Next ‘24 Proprietary 017

 Choose the best RAG for your app
Out of the
Managed Vertex AI Search box

Core
Grounded Generation & Check Grounding API components
On top of: Document AI, Embeddings API, Ranking API and Vector Search

Orchestration

Vertex AI Vertex AI
DIY Reasoning engine RAG API

Google Cloud Next ‘24 Proprietary

Reasoning Engine
1. Deploy your OSS LangChain on Vertex
# Using Reasoning Engine on Vertex AI SDK
When you need orchestration for agent-like behavior, use
can implement them with LangChain
remote_app = reasoning_engines.ReasoningEngine.create(
LangChainAgent(),
2. Support a wide ranges of agent tasks, requirements=[
"google-cloud-aiplatform",
including RAG "langchain",
From simple tasks like information retrieval and RAG, to "google-cloud-bigquery",
complex workflow for customer support ])

response = remote_app.query(query="What colors does the Pixel

3. Full control in development 8 Pro come in?")
Define functions, tools, and workflows, and let Gemini
handle the selection of appropriate API calls and extracting
parameters from prompts.

Now available in Public Preview

Google Cloud Next ‘24 Proprietary
RAG API on Vertex AI # Use RAG as a Tool on Vertex AI SDK

rag_retrieval_tool = Tool.from_retrieval(
1. DIY - Built to be fully customizable
retrieval=rag.Retrieval(
Use default settings to start with, or configure your own source=rag.VertexRagStore(
for every step, from chuck size, chuck overlap, choice of rag_corpora="my-corpus",
embedding models, vector database, and many more.
similarity_top_k=3),
)
2. Enterprises-grade RAG at scale )
# Generate response with RAG
Enterprise-ready features to support large number of
response =
documents, and wide range of connectors of data
GenerativeModel("gemini-pro",tools=[rag_retrieval_tool])
sources
.generate_content("What is RAG and why it is helpful?")

3. Easy, work out-of-box

Managed API enables developers to build RAG
applications with as little as 4 lines of code

Now available in Private Preview

Google Cloud Next ‘24 Proprietary
Grounded
Generation API

Google Cloud Next ‘24 Proprietary 021

Grounded Generation API
Provide data and sources to get grounded answers with citations

Instructions + Query

Sources
Grounded answer
Custom facts with citations with
grounding score
Configuration

Now available in Private Preview

Google Cloud Next ‘24 Proprietary

Grounded Generation API at a glance
Reduces hallucinations & provides citations

Uses a specialized, fine-tuned Gemini model

- Bring your own own text chunks

Easy to add sources: - A Vertex AI Search data store
- Soon: Bring your own retrieval engines

Optimized Prompt-to-search-query generation

You can fully configure retrieval and grounding

Google Cloud Next ‘24 Proprietary
Using the Grounded Generation API
The API takes a query, a
grounding configuration, The API returns a grounded
and custom segments answer with citations

contents { content {
role: "user" role: "model"
parts { parts {
text: "What is the current status of the global economy?" text: "The global economy showed resilience in 2023, but growth is
}
easing due to restrictive monetary policies. Global growth is expected to
}
slow from 5.9% in 2021 to 4.4% in 2022 and further to 2.9% in 2024,
grounding_sources {
according to the International Monetary Fund (IMF)."
grounding_content {
grounding_content_text { }
text: "Fact text #1 goes here" }
attributes { grounding_score: 0.8
key: "url" candidate_citation {
value: "Insert URL here" cited_chunks {
} chunk_text: "Relevant text chunk in the document."
} source: "1"
} source_metadata {
[...]
key: "uri"
}
value: "URI of the document"
grounding_sources {
key: "title"
vertex_ai_search {
search_params { value: "title of the document"
serving_config: "projects/.../..." key: "page_identifier"
} value: "page number of the document"
} }
} }

Google Cloud Next ‘24 Proprietary

Check
Grounding

Google Cloud Next ‘24 Proprietary 025

Why Check grounding

Retrieve &
Rank
Facts Answer Gen Answer
? – Grounding of the response
– Attribution to sources

RAG

Google Cloud Next ‘24 Proprietary

Introducing Check Grounding
Now available in Public Preview

Check Grounding is in Public Preview now

Supports up to ≈400k tokens (200 facts with 10k chars each)

Key use cases:

● During online RAG flow: Generate citations, Highlight
potentially ungrounded claims to the user (or for audits)
● Offline validation use case (model tuning, prompt
engineering)
Upcoming: Support for contradiction scores and generating
contradicting citations
Google Cloud Next ‘24 Proprietary
 For the Builder: Check Grounding example
Confidence
Check
Facts & scores &
Answer gen Answer Grounding
Instructions Attribution to
New!
sources
LLM prompt for the user question:
Tell me about the movie Inception.

Retrieved facts:
● [1] Inception is a 2010 science fiction action film
written and directed by Christopher Nolan, who
also produced the film with Emma Thomas, his
wife. […]
● [2] Inception stars Leonardo DiCaprio, Ken
Watanabe, Marion Cotillard, Joseph Gordon-Levitt,
Elliot Page, Tom Hardy, Cillian Murphy, Tom
Berenger, Dileep Rao, and Michael Caine among
others. […]
● [3] Inception's premiere was held in London on July
8, 2010; it was released in both conventional and
IMAX theaters beginning on July 16, 2010.
Inception grossed over $837 million worldwide,
becoming the fourth-highest-grossing film of
2010. […]
● [4] The movie Inception is about a professional
thief who steals information by infiltrating the
subconscious of his targets.
Google Cloud Next ‘24 Proprietary
 For the Builder: Check Grounding example
Confidence
Check
Facts & scores &
Answer gen Answer Grounding
Instructions Attribution to
New!
sources
LLM prompt for the user question:
Tell me about the movie Inception.

Retrieved facts: Inception was directed by

● [1] Inception is a 2010 science fiction action film Christopher Nolan and he was also
written and directed by Christopher Nolan, who one of the producers.
also produced the film with Emma Thomas, his
wife. […] According to Wikipedia, it is about a
● [2] Inception stars Leonardo DiCaprio, Ken thief who steals information by
getting into people's dreams.
Watanabe, Marion Cotillard, Joseph Gordon-Levitt,
Elliot Page, Tom Hardy, Cillian Murphy, Tom
It received five Academy Awards.
Berenger, Dileep Rao, and Michael Caine among
others. […] It starred Leonardo DiCaprio and
● [3] Inception's premiere was held in London on July Marion Cotillard among others.
8, 2010; it was released in both conventional and
IMAX theaters beginning on July 16, 2010. It made more than $950 million in
Inception grossed over $837 million worldwide, revenue at the box office.
becoming the fourth-highest-grossing film of
2010. […] Correct statements in green, manually added
wrong statements in red.
● [4] The movie Inception is about a professional
thief who steals information by infiltrating the
subconscious of his targets.
Google Cloud Next ‘24 Proprietary
 For the Builder: Check Grounding example
Confidence
Check
Facts & scores &
Answer gen Answer Grounding
Instructions Attribution to
New!
sources
LLM prompt for the user question:
Tell me about the movie Inception.

Retrieved facts: Inception was directed by

● [1] Inception is a 2010 science fiction action film Christopher Nolan and he was also
written and directed by Christopher Nolan, who one of the producers. ✔ Grounded in Fact [1]
also produced the film with Emma Thomas, his
wife. […] According to Wikipedia, it is about a
● [2] Inception stars Leonardo DiCaprio, Ken thief who steals information by ✔ Grounded in Fact [4]
getting into people's dreams.
Watanabe, Marion Cotillard, Joseph Gordon-Levitt,
Elliot Page, Tom Hardy, Cillian Murphy, Tom
It received five Academy Awards.
Berenger, Dileep Rao, and Michael Caine among ✖ Not grounded
others. […] It starred Leonardo DiCaprio and
● [3] Inception's premiere was held in London on July Marion Cotillard among others. ✔ Grounded in fact [2]
8, 2010; it was released in both conventional and
IMAX theaters beginning on July 16, 2010. It made more than $950 million in
revenue at the box office.
✖ Grounded
Inception grossed over $837 million worldwide,
becoming the fourth-highest-grossing film of Overall grounding score:
2010. […] Correct statements in green, manually added
wrong statements in red.
54%
● [4] The movie Inception is about a professional
thief who steals information by infiltrating the
subconscious of his targets.
Google Cloud Next ‘24 Proprietary
Wrap up

Google Cloud Next ‘24 Proprietary 031

Vertex offers a comprehensive
Grounding solution
World knowledge Private knowledge

Google Search Your data RAG RAG Validation

Ground your response in Use Grounded Orchestration & Analysis
the world's knowledge Generation API to
Build your own RAG with: Use Check Grounding to
with Google Search ground in your own data.
RAG API validate grounding of a
grounding for Gemini.
Private Preview generated response.

Public Preview Private Preview Reasoning Engine Public Preview

Public Preview

Google Cloud Next ‘24 Proprietary 032

Ready to build
what’s next?
Tap into special offers
designed to help you
implement what you
learned at Google Cloud
Next.

Scan the code to receive

personalized guidance from
one of our experts.
Or visit g.co/next/24offers
Google Cloud Next ‘24 Proprietary
Thank you

Proprietary 035