Practical Data Science on AWS:

Generative AI

Antje Barth Chris Fregly

Principal Developer Principal Solutions

Advocate - AI/ML Architect - AI/ML

@ AWS - San Francisco @ AWS - San Francisco

https://datascienceonaws.com https://github.com/data-science-on-aws
Data Science
on AWS
datascienceonaws.com
coursera.org/specializations
github.com/data-science-on-aws /practical-data-science

youtube.datascienceonaws.com

meetup.datascienceonaws.com
amazon.com/dp/1492079391
Agenda
Why data science and machine learning in the cloud?

Generative AI use cases

Text-to-image using Stable Diffusion 2.0 and Amazon SageMaker

Source-code generation with Amazon CodeWhisperer

ChatGPT and human-in-the-loop feedback

Evolution of natural language processing (NLP)

Prompt engineering and templates

Generate text using BLOOM/GPT-3 and Amazon SageMaker

Why data science and machine
learning in the cloud?
Scale beyond your laptop and development server

Store & process Large data science

any amount of data and ML toolbox

Elastic, on-demand infrastructure

Purpose-built hardware

Scale up Scale out

Scale down Scale in
Your AWS ML toolbox
Vision Chatbots Business tools Search Healthcare
Rekognition Lex Personalize Kendra HealthLake
Forecast Comprehend Medical
Fraud Detector Transcribe Medical
AI SERVICES Lookout for Metrics
Speech Text Code + DevOps Industrial
Contact centers
Polly Comprehend CodeGuru Panorama Appliance and SDK,
Transcribe Translate Contact Lens DevOps Guru Monitron, Lookout for
Textract Connect Voice ID Equipment, Lookout for Vision

SAGEMAKER STUDIO IDE

Detect bias Visualize in

Label Data Store Pick Train Tune Deploy in Manage Manage edge
and explain
AMAZON data collection prep features notebooks algorithm models faster parameters production & monitor devices
predictions
SAGEMAKER
CI/CD

Deep learning Habana Gaudi

SELF-MANAGED ML AMIs & containers
GPUs
Accelerators
Inferentia Trainium FPGA
Purpose-built hardware
TRAINING

NEW
MACHINE LEARNING

P3DN P4D P4DE DL1 TRN1

NVIDIA V100 NVIDIA A100 NVIDIA A100 HABANA GAUDI AWS TRAINIUM
TENSOR CORE TENSOR CORE TENSOR CORE INTEL
INFERENCE

PLATFORM
PYTORCH
NEW
TENSORFLOW
G4DN G5 G5G INF2
NVIDIA T4 GPU NVIDIA A10G NVIDIA T4G AWS
TENSOR CORE TENSOR CORE INFERENTIA
MXNET
AWS chips optimized for deep learning

Amazon EC2 Amazon EC2

Trn1(n) Inf2
AWS Trainium
High performance BF16/FP16 INT8
BF16/FP16 TF32 FP32
ML training chip, 2.3 Peta FLOPS  4.6 Peta OPS 
3.4 PFLOPS  3.4 PFLOPS  840 TFLOPS purpose-built by AWS

AGGREGATE PEAK MEMORY AGGREGATE NETWORK

ACCELERATOR BANDWIDTH ACCELERATOR CONNECTIVITY
MEMORY MEMORY

512 GB 13.1 TB/sec 384 GB 100 Gbps

EFA NETWORK
AWS Inferentia2
NEURON-CORE V2 High performance NEURON-CORE V2 PYTORCH,
CONNECTIVITY NEURON-LINK V2 TENSORFLOW,
ML inference chip,
800/1600 Gbps
TRANSISTORS PER CHIP
purpose-built by AWS
TRANSISTORS PERV2
NEURON-LINK CHIP SUPPORTED

55,000,000,000 55,000,000,000
Generative AI use cases
What is generative AI?
AI that can produce original content close enough to human generated
content for real-world tasks

Powered by foundation models pre-trained on large sets of data with

several hundred billion parameters

Tasks can be customized for specific domains with minimal fine-tuning

Reduces time and cost to develop ML models and innovate faster

Generative AI use cases

Image Code Text

generation generation generation
Generative AI use cases

Image Code Text

generation generation generation
Diffusion models
1/ “Destroy” the data by gradually adding
small amounts of gaussian noise

2/ “Create” data by gradually denoising a

noisy code from a stationary distribution

Source: https://yang-song.github.io/blog/2021/score
Stable Diffusion

Source: https://jalammar.github.io/illustrated-stable-diffusion/
Stable Diffusion 2.0

Seamless
transformation

4x

Generated by Stable Diffusion 2.0

This interior does not exist Upscaling
DEMO
Generate images from text using Stable Diffusion 2.0
and Amazon SageMaker

https://github.com/aws/amazon-sagemaker-examples/blob/mai
n/introduction_to_amazon_algorithms/jumpstart_text_to_image/
custom_dog_image_generator.ipynb
Generative AI use cases

Image Code Text

generation generation generation
Generate code with
Amazon CodeWhisperer
 Av
ail
ab
Amazon CodeWhisperer code generator le
in
Pr
ev
iew
• Generates code recommendations based on the
comments – and prior code - in your IDE

• Available in popular IDEs such as Visual Studio

Code, JetBrains, AWS Cloud9, AWS Lambda
console

• Supports Python, Java, JavaScript, C#, TypeScript

• Enable CodeWhisperer for your organization with

single sign-in (SSO) authentication

• Sign-up with AWS Builder ID

• Generates open source attribution documentation

for you
DEMO
Generate machine learning code with Amazon CodeWhisperer

https://aws.amazon.com/codewhisperer
Generative AI use cases

Image Code Text

generation generation generation
ChatGPT - Generate product reviews
ChatGPT - Upgrade your code

Source: https://sharegpt.com/c/qBe8ciw
Evolution of natural language
processing (NLP)
Early Natural Language Processing (NLP) models

T=
T=Transformer
Transformer!
 Recent Large Language Models (LLMs)

Nov 2022

Jul 2022

InstructGPT (1.3B) .5, ChatGPT

(between
175B,13.5T)
175B and 13.5T)

Source: State of AI Report 2022, stateof.ai [Slides]

ChatGPT

ChatGPT is fine-tuned from a series of models

trained on text and code called GPT3.5

ChatGPT is further fine-tuned with reinforcement

learning and human feedback to improve results
(a concept introduced with InstructGPT)

Humans are rating the quality of text to ensure

that the models’ goals and values are aligned
with human intent - and reduce toxicity
Reinforcement Learning from Human Feedback (RLHF)

Reinforcement
Learning
Human in
the loop

Human in
the loop
Supervised
Training Reward
Model

Sources: https://openai.com/blog/instruction-following/, InstructGPT Arxiv paper

Example of improvement through human feedback
BEFORE (12/2022) NOW (02/2023)

Source: ChatGPT prompt/response on February 17, 2023

Source: Tweet by Dr. Andrew Ng, December 6. 2022

Prompt engineering
Prompt engineering
A prompt guides the model to generate useful output.

"Who won the most Generated

Prompts can be simple questions:
recent world cup?" LLM text

Prompts can provide context:

Prompts are stateful and combinable
Prompt training dataset for sentiment analysis
Classification

Source: https://huggingface.co/spaces/bigscience/promptsource/blob/main/promptsource/templates/
amazon_polarity/templates.yaml
Prompt dataset for generating/summarizing reviews
Generating

Summarizing
BLOOM/GPT-3 on
Amazon SageMaker
BLOOM/GPT-3

Open source - Responsible AI License

Similar architecture to GPT-3 with improvements
Generates text in 46 natural languages and 13 programming languages
First multilingual LLM trained in complete transparency
BigScience initiative: largest collaboration of AI researchers involved in a single
research project (over 1000 researchers from 70+ countries and 250+ institutions)
Trained in just 117 days (March 11 - July 6, 2022)
Deeply integrated into Hugging Face transformers
BLOOM/GPT-3
DEMO
Generate text from a prompt using BLOOM/GPT-3
and Amazon SageMaker

https://github.com/data-science-on-aws/data-science-on-aws/b
lob/gpt3/00_quickstart/06_Train_and_Fine_Tune_GPT3_PyTorch.
ipynb
Want to learn more?
Data Science
on AWS
datascienceonaws.com
coursera.org/specializations
github.com/data-science-on-aws /practical-data-science

youtube.datascienceonaws.com

meetup.datascienceonaws.com
amazon.com/dp/1492079391
SageMaker Studio Lab: Free ML development in the cloud
https://studiolab.sagemaker.aws

s a nd G PUs
U
Free CP

No credit card,
no AWS account required!
THANK YOU!
https://datascienceonaws.com
https://github.com/data-science-on-aws

Antje Barth
linkedin.com/in/antje-barth

Chris Fregly
linkedin.com/in/cfregly