GENERATIVE AI CON

AMAZON BEDROCK
(WITH REAL EXAMPLES INSIDE 😉)
Guido Nebiolo, AWS Ambassador @ Reply

23 November 2023
WHOAMI
aws sts get-caller-identity

• 20 years developer
• 10 years paid developer
• 8 years paid developer on AWS (mainly)
• 3 years paid to teach developing on AWS
(and other topics)

2
THE AI REVOLUTION

3
WHAT IS GENERATIVE AI?

AI
Generative AI generates new
content for a variety of tasks
ML leveraging pretrained foundation
models that can be customized
with small fractions of data.
DL

Gen AI

4
HOW LLMS WORKS?

PROBABILITY
I am truly excited

5
HOW LLMS WORKS?

PROBABILITY
I am truly excited to (61.91%)
about (21.72%)
for (4.87%)
and (2.93%)
that (1.98%)

6
HOW LLMS WORKS?

PROBABILITY
I am truly excited to be (25.43%)
announce (9.46%)
share (8.18%)
have (7.49%)
see (5.45%)

7
HOW LLMS WORKS?

PROBABILITY
I am truly excited to be a (16.57%)
joining (10.49%)
part (9.70%)
able (7.72%)
working (5.76%)

8
HOW LLMS WORKS?

PROBABILITY
I am truly excited to be a part (82.15%)
member (4.32%)
new (1.03%)
guest (0.88%)
partner (0.34%)

9
HOW LLMS WORKS?

PROBABILITY
I am truly excited to be a part of (99.66%)
\n\n (0.05%)
the (0.04%)
this (0.02%)
o (0.02%)

10
HOW LLMS WORKS?

PROBABILITY
I am truly excited to be a part of the (46.85%)
this (20.92%)
such (4.73%)
a (4.05%)
an (1.52%)

11
HOW LLMS WORKS?

PROBABILITY
I am truly excited to be a part of the team (5.32%)
(1.81%)
amazing (0.83%)
new (0.69%)
community (0.61%)

12
HOW LLMS WORKS?

PROBABILITY
I am truly excited to be a part of the team

13
FMS ON AWS

Managed Managed ML Dev

Out-of-the-Box Proprietary Models
Model-as-a-Service Tooling
Provides a ready-to-use Build GenAI applications Tune or use publicly Build custom models from
solution with predefined on fully managed models available or open-source scratch
configurations, requiring with choice of FMs models as is on managed
minimal setup and model
customization

14
 KEY FEATURES OF BEDROCK

Accelerate development of
generative AI applications
using FMs through an API,
without managing
infrastructure
Choose FMs from Amazon, Privately customize FMs using
AI21 Labs, Anthropic, Cohere, your organization’s data
and Stability AI to find the right
FM for your use case

JURASSIC CLAUDE COMMAND LLAMA SDXL TITAN

Multilingual LLMs for text LLM f or thoughtf ul Text generation model Powerf ul and v ersatile Generation of unique, Text summarization,
generation in Spanish, dialogue, content f or business applications language models that realistic, high-quality generation,
French, German, creation, complex and embeddings model can be used f or a wide images, art, logos, and classif ication, open-
Portuguese, Italian, and reasoning, creativ ity , f or search, clustering, or range of natural designs ended Q&A, inf ormation
Dutch and coding, based on classif ication in 100+ language processing extraction, embeddings
Constitutional AI and languages tasks. Optimized f or and search
harmlessness training dialogue use case

15
EMERGING GENERATIVE AI MODEL PATTERNS
Coherence | context learning

Training your own

#4: Training models model

Model fine-tuning
#3: Model fine-tuning using foundational
models

#2: Retrieval augmented generation (RAG)

In-context learning
using foundational
models
#1: Contextual prompt engineering

Complexity | Time to market

19
PROMPT ENGINEERING
UNDERSTANDING PROMPT ENGINEERING

Summarize the following technical sentence:

Tags: generative ai, security, blogpost
Sentence: Security has been a hot topic since the
birth of Generative AI🔥. From the beginning, AWS
states that security is a shared responsibility
between us and them...
Summary:

21
UNDERSTANDING PROMPT ENGINEERING

INSTRUCTION OUTPUT INDICATOR

Summarize the following
technical sentence

INPUT DATA CONTEXT

22
UNDERSTANDING PROMPT ENGINEERING

INSTRUCTION OUTPUT INDICATOR

Summarize the following Summary:
technical sentence

INPUT DATA CONTEXT

23
UNDERSTANDING PROMPT ENGINEERING

INSTRUCTION OUTPUT INDICATOR

Summarize the following Summary:
technical sentence

INPUT DATA CONTEXT

Sentence: Security has been a
hot topic since the birth of
Generative AI🔥. From the
beginning, AWS states that…

24
UNDERSTANDING PROMPT ENGINEERING

INSTRUCTION OUTPUT INDICATOR

Summarize the following Summary:
technical sentence

INPUT DATA CONTEXT

Sentence: Security has been a Tags: generative ai, security,
hot topic since the birth of blogpost
Generative AI🔥. From the
beginning, AWS states that…

25
UNDERSTANDING PROMPT ENGINEERING

Summarize the following technical sentence:

Tags: generative ai, security, blogpost
Sentence: Security has been a hot topic since the
birth of Generative AI🔥. From the beginning, AWS
states that security is a shared responsibility
between us and them...
Summary:

26
UNDERSTANDING PROMPT ENGINEERING

INSTRUCTION OUTPUT INDICATOR

Summarize the following Summary:
technical sentence

PLEASE
INPUT DATA CONTEXT
Sentence: Security has been a Tags: generative ai, security,
hot topic since the birth of blogpost
Generative AI🔥. From the
beginning, AWS states that…

27
UNDERSTANDING PROMPT ENGINEERING

Please summarize the following technical sentence:

Tags: generative ai, security, blogpost
Sentence: Security has been a hot topic since the
birth of Generative AI🔥. From the beginning, AWS
states that security is a shared responsibility
between us and them...
Summary:

28
 INFERENCE
PARAMETERS

29
INFERENCE
PARAMETERS

TEMPERATURE
Higher the value means more
randomness.

30 Prompt: Captures the beauty of a tropical beach on a hot, sunny day. Include palm trees, crystal-clear waters.
INFERENCE
PARAMETERS

TOP P
Higher the value means it will
only looks at a subset of tokens
whose probability adds up to a
certain threshold (Top P).

31 Prompt: Serene winter wonderland, showcasing a snow-covered forest with glistening trees, a frozen lake, and the peaceful,
cold atmosphere
INFERENCE
PARAMETERS

TOP K
Similar to Top P, but, instead of
working in percentage, it
specifies an absolute number of
tokens.

32 Prompt: Cozy mountain cabin surrounded by a snowy, alpine landscape, with smoke rising from the chimney and a sky full of
stars on a freezing night.
 PROMPT ENGINEERING
TECHNIQUES

33
PROMPT ENGINEERING TECHNIQUES

ZERO-SHOT LEARNING ONE-SHOT LEARNING FEW-SHOT LEARNING

34
ZERO SHOT DEMO
Prompt Engineering

we didn't provide the

model with any
examples of text
alongside their
classifications, the
LLM already
understands
"sentiment"

35
FEW SHOT DEMO
Prompt Engineering

Few-shot prompting
can be used as a
technique to enable
in-context learning
where we provide
demonstrations in
the prompt to steer
the model to better
performance.

36
FEW SHOT DEMO
Prompt Engineering

few-shot prompting
is not enough to get
reliable responses
for this type of
reasoning problem.

37
MORE PROMPT ENGINEERING
TECHNIQUES

CHAIN-OF-THOUGHT GENERATED KNOWLEDGE

enables complex reasoning capabilities through generate knowledge to be used as part of the
intermediate reasoning steps. prompt.

… AND MANY OTHERS

38
RETRIEVAL AUGMENTED
GENERATION
WHY RAG?
Retrieval Augmented Generation

POINT IN TIME HALLUCINATION

FMs knowledge is freezed at the time of of model Generation of text that is not grounded in
training. accurate or real-world information.
40
UNDERSTANDING RAG
Retrieval Augmented Generation

Retrieval Augmented RAG’s internal knowledge

Generation (RAG) is a machine
learning approach that combines
can be easily altered or
elements of both retrieval-based even supplemented on the
models and generative models to fly, controlling what RAG
improve the performance of
natural language understanding
knows and doesn’t know.
and generation tasks.

41
HIGH LEVEL DESIGN
RAG Architecture

High level:
1. Document
Data Vector
Embedder ingestion
Sources Storage
2. Document
retrieval
3. Prompt
augmentation
Retrieval
Prompt Embedder
Engine

Prompt +
Output LLM
Docs

42
UNDERSTANDING RAG
Retrieval Augmented Generation

43
RAG DEMO
Retrieval Augmented Generation

Prompt

Output

44
RAG DEMO
Retrieval Augmented Generation

Prompt + Context

Output

45
FINE TUNING
INTRODUCTION TO FINE TUNING

FOUNDATION TRAINING CUSTOM

MODEL DATA MODEL

47
FINE TUNING WITH BEDROCK

FOUNDATION TRAINING CUSTOM

MODEL DATA MODEL

48
TAKE-AWAYS

• To get better results, give as many details as possible to LLMs.

• Use RAG to cut training costs and decrease TTM when delivering POC
or MVP.
• Consider fine-tuning LLMs instead of giving them too many examples to
learn from.
(How many shots can an LLM handle?)
• Go on and build something, best learning path is hands-one experience.
Be part of the revolution!

49
Q&A
THANK YOU!