Generative AI in Action (MEAP V02)

Amit Bahree
Visit to download the full and correct content document:
https://ebookmeta.com/product/generative-ai-in-action-meap-v02-amit-bahree/
More products digital (pdf, epub, mobi) instant
download maybe you interests ...

The AI Revolution in Project Management: Elevating

Productivity with Generative AI 1st Edition Kanabar

https://ebookmeta.com/product/the-ai-revolution-in-project-
management-elevating-productivity-with-generative-ai-1st-edition-
kanabar/

Creative Prototyping with Generative AI: Augmenting

Creative Workflows with Generative AI (Design Thinking)
1st Edition Patrick Parra Pennefather

https://ebookmeta.com/product/creative-prototyping-with-
generative-ai-augmenting-creative-workflows-with-generative-ai-
design-thinking-1st-edition-patrick-parra-pennefather/

Generative AI for Data Analytics (MEAP) Artur Guja

https://ebookmeta.com/product/generative-ai-for-data-analytics-
meap-artur-guja/

Practical Weak Supervision Doing More with Less Data

1st Edition Tok Wee Hyong Bahree Amit Filipi Senja

https://ebookmeta.com/product/practical-weak-supervision-doing-
more-with-less-data-1st-edition-tok-wee-hyong-bahree-amit-filipi-
senja/
Generative AI: How ChatGPT and Other AI Tools Will
Revolutionize Business 1st Edition Tom Taulli

https://ebookmeta.com/product/generative-ai-how-chatgpt-and-
other-ai-tools-will-revolutionize-business-1st-edition-tom-
taulli/

Generative AI on AWS (Early Release). Antje Barth &

Chris Fregly & Shelbee Eigenbrode.

https://ebookmeta.com/product/generative-ai-on-aws-early-release-
antje-barth-chris-fregly-shelbee-eigenbrode/

Generative AI on AWS: Building Context-Aware Multimodal

Reasoning Applications 1st Edition Fregly

https://ebookmeta.com/product/generative-ai-on-aws-building-
context-aware-multimodal-reasoning-applications-1st-edition-
fregly/

Creating Intelligent Products MEAP V01 Generative AI

advanced analytics smart automation Leo Porter

https://ebookmeta.com/product/creating-intelligent-products-
meap-v01-generative-ai-advanced-analytics-smart-automation-leo-
porter/

Data Storytelling with Python Altair and Generative AI

(MEAP V01) Angelica Lo Duca

https://ebookmeta.com/product/data-storytelling-with-python-
altair-and-generative-ai-meap-v01-angelica-lo-duca/
Generative AI in Action
1. 1_Introduction_to_generative_AI
2. 2_Introduction_to_large_language_models
3. 3_Working_through_an_API_–generating_text
4. 4_From_pixels_to_pictures_–_generating_images
5. 5_What_else_can_AI_generate?
6. index
1 Introduction to generative AI
This chapter covers
Introducing generative AI
Examples of what can be generated using generative AI
Guidance for organizations when adopting generative AI
Understanding key use cases for Generative AI
Comparing Generative AI to other AI

Artificial Intelligence (AI) is familiar and has been around for years. We all have
been using it – when we use a search engine, see a product recommendation,
listen to a curated playlist, or use the suggested words as we type on the phone –
all these are AI-powered scenarios. However, everything seems new in AI today,
with the world on fire talking about AI, specifically Generative AI.

Generative AI, a captivating recent advancement in AI technology, has captured

public interest and sparked global enthusiasm, largely propelled by OpenAI's
ChatGPT, which astoundingly amassed 100 million users within two months – the
fastest user adoption of any technology as of now in humankind. Many enterprises
perceive this technology as the pivotal catalyst for the forthcoming wave of digital
transformation and automation – as it can create new content and processes that
can enhance various business flows. Already, generative AI has been employed to
create a range of content, from images, text, and music to more intricate outputs
such as design patterns and code. Its potential is vast and exciting, capable of
generating almost anything - from new game levels to novel recipes and much
more.

With the rapid advancements of AI technologies, the journey to human-level

performance just got shorter and faster; Figure 1.1. below from Out World in Data
[1] shows how image and language recognition capabilities quickly exceeded
human parity.

Figure 1.1 AI system capabilities [1]

This book will delve into various aspects of this emerging technology. We will
discuss potential use cases and how businesses can incorporate generative AI into
their existing frameworks. We will explore its technical foundations, critical
components, and the modifications required in application architecture. Additionally,
we will present the most effective practices for deploying this technology in a
production environment and managing its operation optimally.

Generative AI fundamentally differs from other types of AI as it is about creating

something new that is not modified or copied from its training data. Where
traditional AI makes predictions based on input data, generative models create new
things by predicting the next set of words from their ability to generate and
understand the semantics of the real world. These new things span a wide range of
text, images, music, and 3D models. We will show how enterprises can adapt to
using these new techniques. Generative AI, akin to an imaginative artist or an
inventive novelist, paints vivid pictures from fragments of information, writes poetry
that touches the soul, and builds whole universes from thin air. Welcome, dear
readers, to the awe-inspiring world of generative AI – a realm where machines
learn to understand, analyze, and create.
1.1 What is this book about?
Upon completing this book, you will get a comprehensive understanding of
generative AI and its applications within an Enterprise. Delving into the
fundamental technologies and principles of generative AI, such as foundational
models, large language models, embeddings, and related algorithms and
architectures, this book will equip readers with the knowledge needed to identify
and implement the necessary steps for incorporating generative AI as part of their
organization’s AI strategies and projects. You will gain insights into the benefits and
challenges that go with the implementation of generative AI.

In addition to the technical underpinnings, this book presents real-world use cases
that show the practical applications of generative AI. Readers will receive guidance
on developing and deploying generative AI models in enterprise settings, covering
topics such as prompts, prompt engineering, model finetuning, and evaluation. We
will also explore emerging application architecture patterns, best practices, and
integration patterns with existing systems and enterprise workflows. As generative
AI continues to evolve, the book will highlight emerging tools and trends that
enterprises should be aware of, including prompt engineering, explainable AI,
transfer learning, and reinforcement learning (specifically reinforcement learning
from human feedback – RLHF).

This book is intended for a diverse audience, including developers, software

architects, data scientists, business decision-makers, and power users working in
enterprises across all sectors. While we may provide industry-specific examples
representing real-world enterprise scenarios, these primarily outline the technical
aspects. Some examples seem like toy examples, but they underpin the core
concepts applicable to enterprises when looking to scale. We aim to demonstrate
how Generative AI can address the distinct challenges many enterprises face,
ranging from governance, internal policies, and regulatory requirements like HIPAA,
GDPR, SOC, PII, and others to data necessities and compliance with regional and
country-specific laws. Nevertheless, the fundamental principles we discuss equally
apply to smaller organizations and startups.

Lastly, readers will gain an understanding of the challenges and risks associated
with generative AI, such as how to use corporate and private data, guard rails and
ethical considerations, data privacy, security, safety considerations, etc., enabling
them to make informed decisions when integrating these technologies into their
organizations.

1.2 What is Generative AI?

Generative AI is not a new field of AI, but it has gained more popularity and
attention lately. It can generate new content in various outputs – from realistic
human faces and writing persuasive text to composing music and developing novel
drug compounds. This new AI technique is about replicating existing patterns,
imagining new ones, crafting new scenarios, and creating 'new' knowledge.

As shown in Figure 1.2, Generative AI is a subsection of AI trained on a vast array

of data, learning the underlying patterns and distributions. The magic lies in its
potential to generate something 'novel' and 'original', a task previously believed to
be the sole domain of human ingenuity.

Figure 1.2 Generative AI Overview

Machine learning and Deep learning

Machine Learning and Deep Learning provide the fundamental techniques we need
to understand before diving into Generative AI. They give us the toolkit to navigate
the landscape of AI, understanding the processes behind data engineering, model
training, and inference. As we progress through this book, we will apply these
principles, but we will not get into the details in this book. Multiple books are
dedicated to both topics, and it would be more prudent for the reader to consult
those for details.

At its simplest, Machine Learning (ML) is the scientific discipline that focuses on
how computers can learn from data. Instead of explicitly programming computers
to carry out tasks, in Machine Learning, we set out to develop algorithms that can
learn from and make predictions or decisions based on data. This data-driven
decision-making is applicable in numerous real-world scenarios, ranging from spam
filtering in emails to recommendation systems on e-commerce platforms.

Deep Learning (DL), a subset of Machine Learning, takes this concept further. It
uses artificial neural networks with several layers – called ‘deep’ learning. These
networks attempt to simulate the behavior of the human brain—albeit in a
simplified form—to ‘learn’ from large amounts of data. While a neural network with
a single layer can still make approximate predictions, additional hidden layers can
help optimize the accuracy. Deep learning drives many AI applications today and
helps execute tasks with improved efficiency, speed, and scale.

Generative AI is an evolution of Deep Learning, as shown earlier. It is a common

mistake where many incorrectly assume ChatGPT is Generative AI. ChatGPT is a
web application that uses generative AI at its simplest level. The rise and popularity
of ChatGPT exposed many folks to Generative AI and the power of these other
generative models called Large language models (LLMs) – which, as the name
suggests, is related to language. OpenAI trained ChatGPT on diverse internet text
to produce a human-like conversation. In addition to ChatGPT, Table 1.1 outlines
some of the key generative AI models used today; these are grouped by generated
AI area types – language generation, image generation, and code generation.

Table 1.1 Popular Generative AI Models

Name Description Area

Generative Pre- Large language model, developed by OpenAI Language
trained and trained on a massive dataset of text and
Transformer (GPT) code. Capable of generating text, translating
languages, writing various kinds of creative
content, and answering your questions
informatively. GPT-4 is the latest version at the
time of this writing.
Llama 2 Meta developed the second version of a Language
natural large language model and has been
open-sourced under a special license.
Claude Claude is a large language model built by a Language
startup called Anthropic. Like OpenAI’s
ChatGPT, Claude, for a given prompt, can
generate text, write code, summarization, and
reasoning, among others.
PaLM2 PaLM (Pathway Language Model) is a large Language
language model developed by Google. It is a
multi-modal model, which means it can
process text, code, and images.
DALL-E Visual AI model developed by OpenAI that can Image
create realistic images from text prompts.
Stable Diffusion Open-source image generation model that Image
generates images from a prompt as input. It is
primarily used to generate detailed images
conditioned on text descriptions and can also
be applied to other tasks such as inpainting,
outpainting, and generating image-to-image
translations.
Midjourney Image generation model using natural Image
language prompts from a startup called
Midjourney, Inc., similar to OpenAI’s DALL-E
and Stable Diffusion.
CodeWhisperer CodeWhisperer is a code generation model Code
from AWS. It can generate code in several
programming languages, including Python,
Java, JavaScript, TypeScript, and others.
CodeLlama CodeLlama is a large language model built on Code
Llama 2 and specifically trained on code. It
also comes in various sizes and supports
multiple popular programming languages.
Codex Large language model trained specifically on Code
code and used to help with code generation. It
supports over a dozen multiple programming
languages, including some of the more
commonly used, such as C#, Java, Python,
C#, JavaScript, SQL, Go, PHP, and Shell,
among others.

Below are a few areas where generative AI is being used today. We expect to see
even more innovative and creative applications as generative AI technology
develops.

Images: Create realistic images of people, objects, and scenes that do not exist
in the real world. This technology is used for various purposes, such as creating
virtual worlds for gaming and entertainment, generating realistic product
images for e-commerce, and creating training data for other AI models.
Videos: Create videos that do not exist in the real world. This technology is
used for various purposes, such as creating special effects for movies and TV
shows, generating training data for other AI models, and creating personalized
video content for marketing and advertising.
Text (Language): Create realistic text, such as news articles, blog posts, and
creative writing. This technology is used for various purposes, such as
 generating content for websites and social media, creating personalized
marketing materials, and synthetic data.
Text (Code): Generative AI models are used to augment and assist developers
when they are writing code. GitHub found in research that developers who use
their Copilot feature feel 88% more productive and are 96% faster on
repetitive tasks.
Music: Generative AI models are being used to create new music that is
original and creative. This technology is used for various purposes, such as
creating music for movies and TV shows, generating personalized playlists, and
creating training data for other AI models.

We’ll dive into the specifics of how generative AI works in the next chapter, but for
now, let’s discuss what you can generate using this technology and how it can help
your enterprise.

1.3 What can we generate?

When it comes to generating things using Generative AI, the sky is the limit; as we
showed earlier, we can generate text, images, music, code, voice, and even
designs. Before we look at some examples of things that can be generated, it is
worth noting that Generative AI does not “understand” the content as humans do.
It uses patterns in the data (part of its training set) to generate new, similar data –
the quality and relevance of the generated content are directly correlated to the
quality and relevance of the training data.

1.3.1 Entities extraction

We can use Generative AI, specifically a Large Language Model (LLM), to extract
entities from text. Entities are pieces of information that are of interest to us. In the
past, we would need to use a Named Entity Recognition (NER) model for entity
extraction; furthermore, that model would need to have seen the data and be
trained as part of its dataset. With LLM models, we can do this without any training,
and it is more accurate. While traditional NER methods were effective, they often
required manual effort and domain-specific customization. LLMs have significantly
reduced this burden, offering a more efficient and often more accurate approach to
NER across various domains. This is a great example of traditional AI being more
rigid and less flexible than generative AI.

In this example, we will use OpenAI’s GPT-4 model to extract the first name,
company name, location, email, and phone number from the text below.

Extract the name, company, email, and phone number from the text below:
 Hello. My name is Amit Bahree. I’m calling from Acme Insurance, Seattle,
WA. My colleague mentioned that you are interested in learning about our
comprehensive benefits policy. Could you give me a call back at (555)
111-2222 when you get a chance so we can go over the benefits? I can be
reached Monday to Friday during normal business hours. If you want, you
can also email me at aweomseinsrance@acme.com. Thanks, Amit.

We can see the entities extracted by the model in the output below in Figure 1.3.

Figure 1.3 Entity extraction

1.3.2 Generating text

Generating text in many ways is the bread-and-butter of Generative AI. We can
generate different things, such as poems, jokes, and personalized content. In this
example, we will use GPT-4 to generate a book launch email for a new book on
Generative AI. We give the model a prompt with our requirements and some
context to build from. We essentially set up the scene, outline the requirements and
another context, such as the text's tone, and it will generate the email for us.

Given the broad nature of the underlying model (called Foundational models –
covered in the next chapter), it is also good at understanding grammar and spelling
mistakes. For example, we misspelled "pracrtices" and "architectes" and it
understood both.

Write a short book launch email for new Generative AI book that is available at
all book stores. The book covers Introduction to Generative AI, what are the
different types and use cases, and best pracrtices for enterprises including how
they can incorporate this new AI technology in their business processes. The
book is published by Manning and the author is Amit Bahree. The name of the
book is "Generative AI in Action: Real-World Applications and Case Studies of
 AI for Enterprises". The target audience is developers and enterprise
architectes and the tone is friendly and exciting.
1. What should be the subject line of the email?
2. What should be the body of the email?

And one gets an answer something like this, which is rather good.

1. Subject Line: "Explore the Power of Generative AI with our New Book:
'Generative AI in Action' by Amit Bahree!"
2. Email Body:
Dear [Recipient's Name],
I am thrilled to announce the release of a remarkable new book, "Generative
AI in Action: Real-World Applications and Case Studies of AI for Enterprises",
authored by Amit Bahree and published by Manning.
Our dynamic business environment is calling for cutting-edge solutions, and
Generative AI is just that. It is one of the most exciting advancements in the
tech industry and can be a game-changer for your business. Whether you're a
developer or an enterprise architect, this book is your ticket to harnessing the
power of Generative AI.
"Generative AI in Action" unravels the complexities of Generative AI. It
provides an accessible introduction and dives deep into its different types, use
cases, and how it can revolutionize business processes. Furthermore, it offers
the best practices for enterprises looking to incorporate this state-of-the-art
technology into their operations.
Here's what you can expect from this enlightening read:
- A comprehensive introduction to Generative AI.
- Exploration of various types of Generative AI and their real-world applications.
- Insightful case studies demonstrating how businesses leverage Generative AI.
- Valuable guidance on the best practices for implementing Generative AI in
your enterprise.
We've carefully curated this book to make Generative AI accessible to anyone
eager to understand and use it. So whether you're new to AI or an experienced
professional seeking to stay at the forefront of technological advancements,
"Generative AI in Action" is the resource you've been waiting for.
We're excited to share that "Generative AI in Action" is now available at all
leading bookstores. Grab your copy today and start your journey towards
transforming your business with Generative AI.
Thank you for your continued support and happy reading!
Best Regards,
[Your Name]
[Your Position]
[Your Contact Information]
1.3.3 Generating Images
We are not restricted to only text; we can generate images using a prompt. To be
clear, the model is not finding a similar image that satisfies these attributes; rather,
it is generating something new that does not exist. It does not matter if we ask for
something whimsical that will not exist in the real world – such as a dog writing a
book with the Taj Mahal in the background. The AI can deconstruct and
comprehend each aspect and then use that to create something new – similar to
how an artist would. In the example below in Figure 1.4, we use OpenAI’s DALL-E 3
model to generate the images from the prompt.

Prompt: "Generate an image of a dog wearing glasses sitting at a table and

authoring a book on AI using a computer. Make it a positive image with the
background of the taj mahal in the window in the distance at the golden hour."

Figure 1.4 Image generation using DALL-E 3

1.3.4 Generating Code

When thinking about generating code, it is helpful not to think of AI as able to
create fully functioning applications but rather for some functions and routines. A
lot of code is about scaffolding of different runtimes and frameworks and less about
the exact business logic. In many of these scenarios, code generation really can
help improve the developer's productivity. In the example below, we use Codex to
generate code for a classic Hello World function. We can give it a prompt such as
the one below, and it will generate the code for us.
 Prompt: Write a hello world equivalent in Python using OpenAI API's for a
developer who is new to using OpenAI and translate the output into French.

And you get an answer like this, including the steps required to start, which is
impressive. Of course, this is just an illustrative example to show the model's power
– understanding the context and rules of the request, including the programming
language, the Software development kit (SDK), and packages to use, etc., and then
generating code. This code does not follow established best practices (e.g., one
should not have their API key in the code).

Listing 1.1 Hello World calling OpenAI service – generated by Codex

import openai

openai.api_key = 'your-api-key' # Replace with your actual OpenAI API key

gpt_model = "text-davinci-003"
# Generate English text
response_english = openai.Completion.create(
engine= gpt_model,
prompt="Hello, World!",
max_tokens=50
)

print(response_english.choices[0].text.strip()) # This prints the completion

of "Hello, World!"

# Translate English text to French

response_french = openai.Completion.create(
engine=gpt_model,
prompt=f'Translate the following English text to French: "
{response_english.choices[0].text.strip()}"',
max_tokens=60
)

print(response_french.choices[0].text.strip()) # This prints the translation

to French

Run the script:

python helloworld.py

We can run this and see the output shown in Figure 1.5 below. Note, given the
non-deterministic nature of AI, we get a little different response each time we run
this. In Chapter 3 – Generating Text, we will see how we can control some of this
using different options and nudge the model towards more deterministic. Some of
these are how large language models (LLMs), a type of Generative AI model, work.
We will explore LLMs in the next chapter.

Figure 1.5 Generated code

While we outline code generation, LLMs can be helpful in other aspects related to
code from explainability, optimization, and readability. We will get into more details
about the code later in Chapter 5.

1.3.5 Ability to solve logic problems

The ability to reason with Generative AI is very impressive. If we can express the
problem domain via a prompt due to the emergent behavior of these AI techniques,
we can get an answer and a step-by-step thought process of the approach. This
resembles how humans think, break down, and approach a problem.

For example, we can ask the model to solve a simple math equation (such as the
example below) and not only solve it but also explain the steps and give us the
answer.

Prompt: Solve this problem and think it through step by step.

If x = 80 and y = 60
What is x, when y = 30?

Here is the response, where we see the model working through its steps and
showing us the exact “thought” process it underwent. Achieving something like this
that can generalize across a wider domain space with traditional AI would be
difficult, if not impossible.

Given that x = 80 when y = 60, we can establish a proportion to solve for x

when y = 30.
In this case, we assume a linear relationship between x and y, that is x is
directly proportional to y. So when y is halved, x would also be halved.
Using the given values:
x1/y1 = x2/y2
Where:
x1 = 80
y1 = 60
y2 = 30
and we're trying to find x2.
We can set up the equation:
 80/60 = x2/30
Solving for x2, we get:
x2 = (80/60) * 30
This simplifies to:
x2 = (4/3) * 30
x2 = 40
So, if y = 30, then x = 40.

Traditional AI cannot solve this problem because it has a narrow understanding of

the world. It cannot understand the question's intent, which is to outline a step-by-
step process. This is because traditional AI is not generative. It can only follow
instructions that are explicitly programmed into it.

Generative AI, on the other hand, can understand the intent of the question
because it has a broader understanding of the world and can generate step-by-step
processes. Another aspect that allows this is the phenomenon of emergent behavior
of generative AI models. This behavior is the ability to outline a step-by-step
process. It is not present in any of the individual components of the model but
emerges from the interaction of the components. The next chapter will cover
emergent behavior in more detail when introducing large language models.

1.3.6 Generating Music

Similar to how we can use prompts and generate images, we can do the same with
music. Music generation is still early compared to text, but there are rapid
advancements in the underlying models and techniques. We can use generative AI
music models such as MusicLM from Google or MusicGen from Meta and others to
generate music. For example, as shown in Figure 1.6, we can use a prompt like
“Create a meditative song that is calming and soothing with flutes and guitars. The
music should be slow and focus on creating a sense of peace and tranquility.” This
generates a couple of 30-second clips.

Figure 1.6 Screenshot of MusicLM

1.4 Enterprise Use Cases
Generative AI is a powerful technology that can create new content or data from
existing ones. Many enterprises use it for different use cases inside and outside
their organizations. Some of these are for improving their workflows or efficiency,
while others provide better services or products to their customers. Depending on
the purpose, the challenges of using Generative AI can vary. For instance, one
common challenge is when the AI system produces something that does not make
sense (i.e., it makes up things – also called hallucinations). This is usually easier to
handle internally than externally because customers might have higher expectations
or lower trust in the AI system.

There are many reasons why an AI system tends to hallucinate, but often because
the underlying model cannot distinguish what facts and fiction are in its training
data. LLMs are trained to generate coherent, context-aware text rather than
factually accurate responses. They tend to hallucinate when the prompt or context
is inaccurate and relevant to the task. We will cover hallucinations and techniques
one can use to minimize them later in the book.

Table 1.2 below outlines a few enterprise use cases – these are generic because
they are more horizontal examples applicable to multiple industries. Table 1.3 later
outlines some industry-specific use cases.

Table 1.2 Horizontal Enterprise Use Cases for Generative AI

Horizontal Use Case Description

 Content Generation GPT-4 can generate content like blogs, reports, emails,
and social media posts. This can help businesses to scale
their content marketing efforts.
Personalized Generative AI can create personalized marketing content
marketing like emails, landing pages, and social media posts. This
can help businesses to reach their target audience more
effectively and increase conversion rates.
Customer service Generative AI can be used to create chatbots that can
answer customer questions and resolve issues. This can
free human customer service representatives to focus on
more complex tasks. Many of these are being
implemented via an Enterprise ChatGPT-like paradigm.
Risk management Generative AI can identify and predict risks like fraud,
cyberattacks, and supply chain disruptions. This can help
businesses to mitigate risks and protect their assets.
Compliance Generative AI can generate compliant documents, such as
contracts, reports, and disclosures. This can help
businesses to save time and money and reduce the risk of
non-compliance.
Software In software development, generative AI can generate new
Development code, provide code snippets, or even write simple
software, potentially saving time and reducing errors. In
addition, it also helps document code, refactor, generate
test cases, and optimize existing code.
Data Augmentation In data science and machine learning, generative AI can
create synthetic data when there is insufficient real data
for model training.
Contract management A key use case for Generative AI is contract generation
and contract understanding – both creating draft legal
documents and understanding legal contracts that
incorporate specific regulatory and regional legal
requirements for enterprises and specific corporate
policies – helping reduce human mistakes and enterprises
make an informed decision.

Table 1.3 Industry-specific Use Cases for Generative AI

Industry Use Case Description

Financial In the financial sector, generative AI can simulate
different scenarios to help with decision-making, risk
model assessment, and the development of new financial
products and services. Customer operations to enhance
 services and resolutions for each client based on
transactions and history.
Healthcare Generative AI is used to develop new drugs and
treatments, design medical devices, create personalized
patient treatment plans, and generate patient
documentation on instructions, risks, and drug
interactions.
Manufacturing Generative AI is used to design new products, optimize
manufacturing processes, and improve quality control.
Retail and CPG Generative AI is used to personalize shopping
experiences, recommend products, manage inventory,
accelerate consumer research, enhance the supply chain,
etc.
Marking and Sales Marketing and Sales already use a lot of narrow AI.
Generative AI is helping enhance some aspects of the
industry – specifically helping understand real-time
customer trends, personalized outreaches embedded into
virtual assistants, and dynamic customer journeys, as
some examples.

These are just a few enterprise use cases for generative AI. As the technology
continues to develop, we can expect to see even more innovative and impactful
applications for generative AI in the years to come.

When not to use Generative AI?

Using generative AI in an enterprise environment is quite different from folks using

it for personal reasons. Organizations must follow certain regulations and
expectations that apply to their industries, some of which will vary by location. In
addition, there are other financial, legal, technical, and moral things to consider.
Say, if the AI-generated content is incorrect, full of bias, or just plain offensive –
what issues would that cause? For example, there is not much harm if an AI
chatbot messes up a recipe, but it is a whole different story if it gives bad
instructions to someone fixing a complex piece of machinery.

If enterprises do not set up proper ethical guidelines for generative AI, it could lead
to unintended consequences. Generative AI can create misinformation and fake
content, including fake news articles, fake images, sensitive content, and malicious
content. A picture is not worth a thousand words anymore; some images are so
good that it is getting harder to distinguish fake from real images. In some cases,
the generated output could also infringe on third-party copyrighted material. Adding
human oversight and contribution can help address some of these challenges.
Generative AI models can also unintentionally amplify biases present in training
data, leading to problematic outputs that perpetuate stereotypes and harmful
ideologies.

Companies need a solid plan for using generative AI and ensuring it aligns with
their business goals, like how it will affect sales, marketing, commerce, service, and
IT jobs. And where there are life-and-death decisions, ensure there is a human-in-
the-loop who is making the final decision with the AI as a co-pilot and assisting.

1.5 How is Generative AI different from

Traditional AI?
Traditional AI, which some now refer to as Narrow AI, is predominantly the
common form of AI we encounter today. Traditional AI uses classical data science
and a systematic approach to extracting insights and knowledge from data using
various methods and techniques. It typically involves the following steps: data
collection, data preparation, data analysis, feature engineering, training, and data
validation. It is used for predicting things operating in pre-established boundaries
that it has been trained on. These boundaries are the rules and instructions that are
coded in a model. It can only act based on predefined conditions, constraints, and
potential outcomes. The outcome, thus, is deterministic and relatively predictable.
For instance, a traditional AI could be programmed to automate customer service
through chatbots by generating responses based on predefined scripts. While these
systems are highly effective within their prescribed domain, they stay within their
swim lanes and cannot learn and adapt to situations outside their programmed
knowledge – hence the “narrow” aspect. Almost all the AI implementations and
implementations enterprises have done thus far are traditional AI.

On the other hand, generative AI leans toward a probabilistic approach, where the
outcome is calculated based on probabilities influenced by the input data and
learned patterns. This allows these AI systems to create outputs that were neither
hard-coded nor explicitly taught to the system. Generative AI needs the classical
data science process, enhancing and complementing it in many ways. Generative AI
can help deal with new types of data and content, evaluate the quality and validity
of generated outputs, and ensure the ethical and responsible use of generative AI.
Classical data science and generative AI must work together to create value and
impact from data.

One significant difference is that the modality to “talk” to these newer Generative
AI models is by prompting them via a prompt, as we have seen in examples earlier.
More formally, a prompt is a set of instructions that tells the generative AI system
what kind of content we want to create. The better the prompt, the better the
results. A prompt should be tailored to the type of response you want to receive
and to the specifics of the Generative AI you use. We will cover Prompt engineering
in more detail later.

Prompting, in some ways, allows us to be more expressive and not only outline the
requirements and the intent but also capture empathy and emotion via language.
Prompt engineering is a new rising area for developers and enterprises – some of
which are more art and less science.

Prompt: Write a funny haiku about prompt engineering vs. traditional AI.

AI generated text:

Old AI's a bore,

With prompts we engineer more -
Laughs galore, encore!

Table 1.4 outlines broad categories to show differences between generative and
traditional AI architectures. It is also important to note that there is a lot of overlap
between different types of AI. For example, a chatbot might use both generative
and predictive models - generating responses based on a learned understanding of
language and predicting what type of response a user will likely find most helpful.

Table 1.4 How generative AI differs from traditional AI

Area Difference
Creation Traditional AI focuses on prediction or classification tasks, identifying
vs what something is or forecasting what will happen next based on
Prediction existing data. Generative AI creates new content and outputs that did
not exist in the original data.
Hosting Relative to generative AI, traditional AI models are less complex and
and require fewer computing resources, allowing them to run on various
Inference hardware, from small edge devices to large cloud clusters and
everything in between. This flexibility cloud-to-edge is a huge
advantage for enterprises.
Generative models are very large and complex; for the most part,
they are available only on large cloud compute nodes via an API.
These have other advantages where the knowledge of the world
encoded in these foundational models is available to everyone, and
other constraints that one needs to consider.
Note: There is a growing set of smaller open-source models that can
be run on-premise, but today, they are still more experimental and
nascent. Many claim to be AI Quality (AIQ), similar to OpenAI’s
models. Given the broad spectrum that generative AI covers, there is
 a new emerging set of benchmarks, such as Stanford’s Holistic
Evaluation of Language Model (HELM -
https://crfm.stanford.edu/helm/latest/) and Hugging Face recently
published an Open LLM Leaderboard -
https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard.
Training Generative models require a different method of training (self-
method supervision and multi-task learning), which is longer and much more
expensive because of the massive scale of data, model sizes, and
computing resources needed. The costs and complexity of managing
this are enormous, and we will touch on them later in the chapter.
Training Generative AI models are typically trained on large datasets of
dataset existing content, while traditional AI models are typically trained on
smaller datasets of labeled data. For example, a generative AI model
for image generation might be trained on a dataset of millions of
images. In contrast, a discriminative AI model for image classification
might be trained on a few thousand labeled image datasets.
Model Generative AI models are often more complex than other types of AI
complexity models. This is because generative AI models need to learn the
patterns and relationships in the data to generate new content similar
to the existing content.
Adaptation There are no adaptive techniques for traditional AI other than labeling
approach more data and going through a full ML loop of training, deploying, and
evaluating. Generative AI, on the other hand, has world knowledge
and is vast. Sometimes, one needs to tailor them to specific needs
and tasks or distill internal private and proprietary knowledge; this is
done via adaptation. There are various techniques depending on the
needs.

1.6 What approach should Enterprises take?

Generative AI can offset the limitations of traditional data-driven AI systems and
offer an extraordinary toolkit to build smarter applications and systems; for
enterprises, the possibilities are vast and exciting. Imagine designing systems that
self-optimize over time or developing AI assistants (Copilots) that can draft emails,
reports, generate code snippets, etc., thereby improving productivity and reducing
the time-to-market.

Many organizations may fall for the excitement and fear of missing out on
generative AI, which appears to be magical. However, the truth is that having a
foundational model like GPT-4, a big language model, does not make any difference
by itself. These advanced AI systems must be implemented and connected to the
enterprise's business lines and processes like any other external software. We will
see examples of how to implement this in subsequent chapters.

At a high level, there should not be many changes from an overall approach; it is
still advised that enterprises should take a thoughtful and strategic approach when
incorporating Generative AI. Below are a few key considerations – these span
various dimensions that most enterprises need to consider, from strategic, business,
and technical.

Crawl, walk, and run – Start small and do not rush in to do too much too soon.
Start with a small pilot project to evaluate, learn, and adapt. This is a complex
technology, and it takes time to develop and deploy effective generative AI
applications. Do not expect to see results overnight.
Define clear objectives and the right use cases – It is important for enterprises
to carefully evaluate potential use cases and select those that are most likely to
deliver value. The selected use case will guide the choice of AI models, data
preparation, and resource allocations. Some generative AI applications are
more mature and have a proven record of success; for others, it is still early
days.
Establish governance policies – Generative AI can generate data, some of
which may be sensitive or harmful. Enterprises must establish governance
policies to ensure this data is used responsibly and securely. These policies
should address issues such as data ownership, data privacy, and data security.
Establish Responsible AI and Ethical Governance – Considering the ethical
implications of using generative AI is important. Establish a separate
responsible AI and ethical set of policies that reflect the company's values and
that it would be important to manage the reputation and the brand. This
includes concerns around bias in AI outputs, the potential misuse of generated
content, hallucinations and incorrect details in generated content, and the
implications of automating tasks that humans previously performed. A robust
AI governance and ethics framework can help manage these risks.
Experiment and iterate – Unlike computer science, AI, specifically generative
AI, is nondeterministic, and depending on the model parameters and settings,
the output can be quite different. As with any AI application, it is essential to
take an iterative approach when implementing generative AI. Start with smaller
projects, learn from the outcomes, and gradually scale up. This approach helps
to manage risk and gain practical experience.
Design for failure – Most generative AI models today are commercially available
as a cloud API. As such, they are complex and have a considerable latency
compared to more traditional APIs. Enterprises should adhere to cloud best
practices and design for failure. Enterprises should factor in best practices of
retry mechanics, including exponential backoff policies, caching, security, etc.
Expand existing Architecture – These new generative AI endpoints are just
additional pieces of the overall system. As such, most organizations will want to
 keep their existing architecture guidance and practices and expand their
existing architecture and best practices, not start from scratch. There are new
constructs that need to be incorporated – things such as context windows,
tokens, embeddings, etc.
Manage cost – Generative AI is complex and much more expensive. The cost is
typically measured differently (such as tokens) and not API calls. Much of this
is new and different for enterprises, and the costs are quite easy to get out of
hand.
Complement traditional AI – In most cases, Generative AI would help assist
existing investment in traditional AI enterprises already have. Both sets of
technologies are not mutually exclusive but rather support each other.
Open-Source vs. Commercial models – Some models are commercially
available, such as Azure OpenAI’s GPT models, and open source, such as
Stable Diffusion. Depending on the use case, it is important to validate which
models to use, what the licensing allows, and what legal and regulatory
aspects are already covered.

1.6.1 Architecture Considerations

If you are an enterprise developer seeing all the news on Generative AI and the
various product announcements from major technology companies, you might think
for AI, everything has changed. Still, the reality is that nothing has changed.

From an enterprise perspective, there are new aspects of Generative AI that one
needs to consider – most, if not all, of these would be things to add to existing
architecture best practices and guidance and not throw out anything. We will cover
the details later in the book, but new architectural patterns must be accounted for
at a high level. Many of these we have touched on earlier in this chapter, but the
key ones are:

Prompts, and later, we will see how to consider engineering and managing
aspects around prompts, including tokens and context windows.
Model adaptation to make the output better for specific tasks.
Integrating generative AI into existing enterprise line-of-business systems –
these new AI models alone do not solve a business problem.
Design for failure – something that is not new per se when building mission-
critical systems, but many still take shortcuts.
Cost and ROI – These generative AI systems are tremendously expensive, a
key part because the underlying compute is very expensive. The costs will
come down over time, but they must be conscious, planned, and designed
upfront. For example, the cost of GPT3.5 Turbo from OpenAI came down by
90%, and its quality went up by 90% compared to GPT-3.
 Implement policies and approaches for Open-source (OSS) vs. Commercial
models. Each week, newer models are powering generating AI systems being
released – some are commercial, others are OSS, with different licensing
structures.
Vendor – There are a few vendors in production that enterprises can use today,
but more are coming soon. Today, two of the most mature are OpenAI and
Azure OpenAI –similar where the former targets smaller companies and
startups, with the latter more at Enterprises. Google is also releasing its
Generative AI suite on Google Cloud, and there have been announcements
from Amazon. In addition, there are also a lot of well-funded startups that have
announced similar products. Enterprise would want to look at each one in the
complete sense of the vendors and identify which one they would want to take
a dependency on.

1.7 So, your enterprise wants to use

generative AI...what now?
Your enterprise has taken a critical step towards leveraging generative AI to drive
innovation and efficiency. However, understanding what comes next is crucial to
maximizing the benefits and mitigating this advanced technology's risks.

To help get started, we will use the example of implementing an “Enterprise

ChatGPT” and outline the steps that are needed at a high level. Throughout the
next few chapters, we will dig into more technical details, including guidance on
implementing and best practices. Figure 1.7 below shows a high-level overview of
what a typical workflow in an enterprise might look like.

Figure 1.7 High-Level overview of implementing Generative AI

First, you need to define your goals clearly. What problems are you trying to solve
with generative AI? Where can it add the most value? This could be anything from
content creation in marketing to improving customer service with chatbots,
predictive modeling for business strategies, or even product or service innovation.
We are implementing an Enterprise ChatGPT like OpenAI’s ChatGPT in our example.
Still, one that is deployed and runs in an enterprise setting, using internal and
proprietary data, and only authorized users would have access to it. The Enterprise
ChatGPT,

Second, we need to ensure you have the right resources in place – this includes
people with the right skills, the appropriate hardware and software infrastructure,
outlining measures of success, and setting up the proper governance and ethics
guidelines.

Third, consider the data. In our example, the Enterprise ChatBot would need access
to the enterprise data, which is relevant, high-quality, and can be used by the user.
This data needs to be ingested and indexed so we can use it to help answer
proprietary questions. And before that, the data must be managed properly,
ensuring privacy and legal compliance. Remember, the quality of the data fed will
influence the output quality.

Next, we need to integrate the Enterprise ChatBot into the line of business
applications that address the use case and the problem we are trying to address. As
an enterprise, we will also want to address the risks associated with generative AI
and implement corporate guidance around Safety and Responsible AI.

Lastly, even though we might be ready to deploy in production, implementing

generative AI is not a one-time event but a journey. It requires continuous
monitoring, testing, and fine-tuning to ensure it works optimally and responsibly.
It's a good idea to start with smaller, manageable projects and gradually scale up as
you gain more confidence and expertise in handling this powerful technology.

Adopting generative AI is a significant commitment that could transform your

enterprise, but it requires careful planning, appropriate resources, ongoing
monitoring, and an unwavering focus on ethical considerations. With these in place,
your enterprise can reap the manifold benefits of generative AI.

1.8 Summary
Generative AI can be used for multiple use cases – Entity extraction,
generating specific and personalized text, generating images, generating code,
interpreting, solving logical problems, and generating music.
Generative AI use cases can be horizontal across most industries (such as
customer services and personalized marketing) or industry-specific (such as
fraud detection in finance or personalized treatment plans in healthcare).
 Traditional AI predominantly operates in pre-defined narrow swim lanes and
can act only in those dimensions, unlike generative AI, which is broader and
allows for more flexibility.
Outlined an approach and architecture considerations for enterprises to adopt
and implement generative AI.

1.9 References
[1] M. Roser, "The brief history of artificial intelligence: The world has changed fast
– what might be next?," 6 12 2022. [Online]. Available:
https://ourworldindata.org/brief-history-of-ai.
2 Introduction to large
language models
This chapter covers
Overview of Large Language Models (LLMs)
Key use cases powered by LLMs
Foundational Models and their impact on AI Development
New architecture concepts for LLMs - prompts, prompt
engineering, embeddings, tokens, model parameters, context
window, and emergent behavior
Comparison of open-source and commercial LLMs

Large Language Models (LLMs) are Generative AI models that can

understand and generate human-like text based on a given input.
LLMs are the foundation for many natural language processing (NLP)
tasks, such as search, speech-to-text, sentiment analysis, text
summarization, and more. LLMs are general-purpose language
models that are pre-trained and can be fine-tuned for specific tasks
and purposes.

This chapter explores the fascinating world of Large Language

Models (LLMs) and their transformative impact on artificial
intelligence. As a significant advancement in AI, LLMs have
demonstrated remarkable capabilities in understanding and
generating human-like text, enabling various applications across
various industries. We dive into the critical use cases of LLMs, the
different types of LLMs, and the concept of foundational models that
have revolutionized AI development.

Throughout the chapter, we discuss essential concepts in LLMs, such

as prompts, prompt engineering, embeddings, tokens, model
parameters, context windows, transformer architecture, and
emergent behavior. Finally, we compare open-source and commercial
LLMs, highlighting their advantages and disadvantages. By the end
of this chapter, you will have a comprehensive understanding of
LLMs and their implications for AI applications and research. LLMs
are built on foundational models; we will start by outlining what
these foundational models are before we get into the details of
LLMs.

2.1 Overview of Foundational

Models
Introduced by Stanford researchers in 2018, foundational models
have significantly transformed AI systems construction. They diverge
from task-specific models, shifting to broader, more adaptable
models trained on large data volumes. These models can excel in
diverse natural language tasks, such as machine translation and
question answering, as they learn general language representations
from extensive text and code datasets. These representations can
then be used to perform various tasks, even tasks they were not
explicitly trained on, as shown in Figure 2.2 below.

In more technical terms, foundational models utilize established

machine learning techniques like self-supervised learning and
transfer learning, enabling them to apply learned knowledge across
various tasks. Developed using deep learning, these models use
multilayered artificial neural networks to comprehend complex data
patterns, hence their proficiency with unstructured data like images,
audio, and text.

Note:

Transfer learning is a technique in machine learning where a model

developed for one task is reused as a starting point for a similar
task. Instead of starting from scratch, we use the knowledge from
the previous task to perform better on the new one. It's like using
knowledge from a previous job to excel at a new but related job.

Generative AI and foundational models are closely interlinked. As we

outlined, foundational models, trained on massive data sets, can be
adapted to perform various tasks; this property makes them
particularly suitable for generative AI and allows for creating new
content. The broad knowledge base of these models allows for
effective transfer learning, which can be leveraged to generate new,
contextually appropriate content across diverse domains. They
represent a unified approach, where a single model can generate
various outputs, offering state-of-the-art performance due to their
extensive training. Without foundational models as the backbone, we
won't have any generative AI models.

Figure 2.1 Foundational model overview

Some examples of the more common foundation models are listed
below.

GPT (Generative Pre-trained Transformer) is a natural language

processing model developed by OpenAI. It is a large language
model, which means it has been trained on a massive dataset of
text and code. This makes it capable of generating text,
translating languages, writing creative content, and answering
your questions informatively. GPT-4, the latest version at the
time of this writing, is also a multi-modal model – it can manage
both language and images.
Codex is a large language model trained specifically on code
and used to help with code generation. It supports over a dozen
multiple programming languages, including some of the more
commonly used, such as C#, Java, Python, C#, JavaScript, SQL,
Go, PHP, and Shell, among others.
Claude is a large language model (LLM) built by a startup called
Anthropic. Like OpenAI's ChatGPT, Claude predicts the next
token in a sequence when given a certain prompt and can
generate text, write code, summarization, and reasoning.
BERT (Bidirectional Encoder Representations from Transformers)
is a natural language processing model developed by Google. It
is a bidirectional model, meaning it can process text in both
directions, from left to right and right to left. This makes it
better at understanding the context of words and phrases.
PaLM (Pathway Language Model) and its successor PaLM2 are
large multimodal language models developed by Google. The
multi-modal model can process text, code, and images at the
same time. This makes the single model capable of performing
a wider range of tasks across those modalities, compared to
traditional language models operating only in one modality.
LaMDA (Language Model for Dialogue Applications) is a large
language model developed by Google. It is designed for
dialogue applications, such as chatbots and virtual assistants. It
can understand natural language and respond in an informative
and engaging way.
Once a foundational model is trained, it can be adapted to a wide
range of downstream tasks by fine-tuning its parameters. Fine-
tuning involves adjusting the model's parameters to optimize the
model for a specific task. Fine-tuning can be done using a small
amount of labeled data. By fine-tuning these models on specific
tasks or domains, we leverage their general understanding of
language and supplement it with task-specific knowledge. The
benefits of this approach include time and resource efficiency,
coupled with remarkable versatility. We can also adapt a model via
Prompt engineering, which we discuss later in this chapter. Now that
we have understood foundational models let us explore LLMs.

2.2 Overview of LLMs

LLMs are a significant advancement in AI and are trained on a vast
amount of text data, such as books, articles, and websites, to learn
patterns in human language. They are also hard to develop and
maintain, requiring lots of data, compute, and engineering
resources. OpenAI's ChatGPT is an example of an LLM – it generates
human-like text by predicting the probability of a word given the
previous words used in the text.

The model learns to generate coherent and contextually relevant

sentences by adjusting its internal parameters to minimize the
difference between its predictions and the actual outcomes in the
training data. When generating text, the model chooses the word
with the highest probability as its subsequent output and then
repeats the process for the next word, and so on.

Large language models are foundational models that have been

adapted for natural language processing and natural language
generation tasks. These large language models are general-purpose
and can handle tasks without task-specific training data. As we
briefly saw in the previous chapter, given the right prompt, they can
answer questions, write essays, summarize texts, translate
languages, and even generate code. LLMs can be applied to many
applications across different industries, as outlined in Chapter 1 –
from Summarization, Classification, Q&A Chatbots, Content
Generation, Data Analysis, Entity Extraction, and more. Before we
get into more details of LLMs, let us look at the Transformer
architecture – which makes these foundational models possible.

2.3 Transformer Architecture

Transformers are the bedrock of foundational models responsible for
their remarkable language understanding capabilities. The
Transformer model was first introduced in the paper "Attention is All
You Need" by Vaswani et al. in 2017. Since then, Transformer-based
models have become state-of-the-art for many tasks. GPT and BERT
are examples of Transformer-based models; the "T" in GPT stands
for Transformers.

At their core, Transformers use a mechanism known as attention

(specifically self-attention), which allows the model to consider the
entire context of a sentence, considering all words simultaneously
rather than processing the sentence word by word. This approach is
more efficient and can improve results on many NLP tasks.

The strength of this approach is that it captures dependencies

regardless of their position in the text, which is an essential factor in
language understanding. This is key for tasks such as machine
translation and text summarization, where the meaning of a
sentence can depend on words that are several words away.

Transformers can parallelize their computations, which makes them

much faster to train than other types of neural networks. This
mechanism enables the model to pay "attention" to the most
relevant parts of the input for the task.
In the context of generative AI, a transformer model would take an
input (such as a prompt) and generate an output (like the next word
or the completion of the sentence) by weighing the importance of
each part of the input in generating the output. For example, in the
sentence "The cat sat on the...", a Transformer model would likely
give much weight to the word "cat" when determining that the likely
next word might be "mat". These models exhibit generative
properties by predicting the next item in a sequence—the next word
in a sentence or the next note in a melody. We explore more of this
in Large Language Models in the next chapter.

Transformer models are usually very large, requiring significant

computational resources to train and use. Using a car analogy, think
of Transformer models like supercharged engines that need much
power to run but can do amazing things. Think of them as the next
step after models like ResNET 50, which is used for recognizing
images. While ResNET 50 is like a car with 50 gears, OpenAI’s GPT3
is like a mega-truck with 96 gears and extra features. Because of
their advanced capabilities, these models are a top pick for creating
intelligent AI outputs.

LLMs use Transformers, which are composed of an encoder and a

decoder. The encoder processes the input text (i.e., the Prompt) and
generates a sequence of hidden states that represent the meaning
of the input text. The decoder uses these hidden states to generate
the output text. These encoders and decoders form one layer -
thinking of them as a "mini-brain". In Figure 2.3 below, we show one
layer of this encoder-decoder, but in reality, multiple layers can be
stacked one upon another. As outlined earlier, GPT3 is a decoder-
only model with 96 layers.

2.3.1 Training Cut-Off

In the context of foundational models, the "training cut-off" refers to
the point at which the model's training ends, i.e., the time until the
data used to train the model was collected. In the case of AI models
developed by OpenAI, such as GPT-3 or GPT-4, the training cut-off is
when the model was last trained on new data.

This cut-off is important because, after this point, the model is not
aware of any events, advancements, new concepts, or changes in
language usage that occur. For example, the training data for GPT
cut-off in September 2021, meaning the model does not know real-
world events or advancements in various fields beyond that point.

The key point is that while these models can generate text based on
the data they were trained on, they do not learn or update their
knowledge after the training cut-off. They cannot access or retrieve
real-time information from the internet or any external database.
Their responses are generated purely based on patterns they have
learned during their training period.

Note:

With the more recent announcements that the premium versions of

ChatGPT will have access to the internet via the Bing plugin, it
doesn’t mean that the model has more up-to-date information. This
uses a pattern called RAG – Retrieval-Augmented Generation, which
we will cover later in the book in Chapter 7.

2.4 Types of LLMs

There are three categories of LLMs, as shown in Table 2.1 below.
When we talk about LLMs, having the context in which one is talking
is crucial– it might not be evident in some cases. This is crucial, as
the path one goes down when using the models isn't
interchangeable, and picking the right type depends on the use case
one tries to solve. Furthermore, there is also a dependency on how
effectively one can adapt the models to specific use cases.
Table 2.1 Types of LLM

LLM Type Description

Base LLM These are the original, pre-trained models trained
on a massive corpus of text data and can generate
text based on the patterns they learned during this
training. Some also call these "Raw" Language
models or even refer to them as Foundational
models; they can be used "out of the box" to
generate text. They learn powerful and general
representations but lack specific expertise in a
particular task. GPT3's DaVinci model is an example
of a base LLM.
Instruction- This involves using a base LLM and providing
based LLM explicit instructions in prompt input. Many examples
we saw in the previous chapter follow this – where
we instruct the model to follow. For example,
"Translate the following text to French: ..." or
"Summarize this article: ...".
Fine-tuned Fine-tuning involves taking the base LLM and
LLM further training it on a task it might perform poorly
at, often in a specific domain. For example, training
the model on medical literature if we want it to
understand medical topics or training it on customer
service interactions if you want it to respond to
customer inquiries for a specific industry. Fine-
tuning can help make the model more accurate or
helpful to particular tasks or domains, but it
requires additional data and training time.

It's worth noting that all these methods have their pros and cons:

Base LLM are versatile and can handle many tasks without
additional training. However, it might not be as accurate or
reliable as you'd like for specific tasks or domains – especially in
an enterprise setting.
 Instruction-based usage can be very effective for some tasks
but requires careful prompt crafting and doesn't fundamentally
change the model's capabilities. This is where many of the
prompt engineering techniques and best practices apply.
Fine-tuning can yield excellent results for specific tasks or
domains. However, it requires additional resources and comes
with the risk of overfitting the training data, which could limit
the model's ability to generalize to new examples.

In addition, one can take approaches (zero-shot, few-shot, and

transfer learning) to further adapt the LLM for a specific task to
make them perform better and be more robust in those specific
domains. To some extent, the type of LLM implemented also dictates
which approach is better suited for the need. Each approach has its
strengths and weaknesses, and the best approach depends on the
specific task, the data available, and the resources at hand.

Zero-shot Learning: A model can generalize to a task without

seeing examples of that task during training. For example, we
could ask a model trained only in English to translate English to
German, even if it hasn't seen any specific examples of the
German language during training. It can do this zero-shot
translation using semantic similarity, word embeddings, and
machine learning. Using these techniques, we can measure how
similar two words are even in different languages.
Few-shot Learning: This involves showing the model a few
examples of the task we want to perform and then asking it to
do the same task with a new example. For example, if you want
a model to identify the sentiment of a tweet, we might show it a
few examples of tweets and associated sentiment and then ask
it to predict the sentiment of a new tweet.
Transfer Learning involves training a model on one task and
then using what it learned for a different but related task. For
example, even though LLMs have been trained in language and
not specific customer support tickets, they can be used to
categorize customer support tickets into different categories like
 billing, technical issues, or general inquiries. This can help
streamline the customer support process and ensure a speedy
resolution, with higher customer satisfaction.

2.5 Open-Source vs. Commercial

LLMs
Today's commercial models provide top-notch performance
regarding AI quality and wide-ranging abilities. However, since the
release of ChatGPT, there has been a significant shift towards open-
source models. Many of these open-source initiatives focus on
developing smaller foundational models, asserting they can achieve
nearly the same quality levels without significant loss. Figure 2.2 [1]
below shows a view of these lineages and how they have exploded
quickly.

Figure 2.2 Timeline of LLMs with 10+B parameters – A Survey of Large

Language Models [1]

2.5.1 Commercial LLMs

Several commercial LLMs exist as of this going to print and have
started making traction. Almost all of them follow the OpenAI
paradigm and are exposed via an API we use. Many of them, while
still being a startup, have serious funding and deep research
background of the founders.

OpenAI: OpenAI is an AI research lab that develops and

publishes cutting-edge AI models, such as the GPT series. They
certainly are the ones with the most mindshare today. They
have several foundational models, such as GPT-4, DALL.E, and
ChatGPT. Today, it is the most mature in this group, with serious
backing and ownership by Microsoft.
Azure OpenAI: Azure OpenAI and OpenAI offer access to
powerful language models but differ in their nature and
implementation. The primary distinction is that Azure OpenAI is
a managed service, whereas OpenAI is not. Microsoft handles
the underlying infrastructure and maintenance for Azure
OpenAI, making it a valuable option for businesses lacking the
resources to manage their OpenAI deployment. Furthermore,
Azure OpenAI packages the raw OpenAI models into developer-
friendly services that developers can seamlessly integrate into
their applications. These services run on Azure, ensuring high
scalability, reliability, and global availability.
Anthropic: A startup founded by ex-OpenAI engineers has
released Claude – an LLM that can generate text and code.
Their key differentiator is implementing the LLM using
Constitutional AI [2]. Constitutional AI uses Reinforcement
Learning (RL) and traditional supervised learning and claims to
produce less harmful outputs. As of this publication, Anthropic
was backed by both Google and Amazon.
Google: The LaMDA (Language model for dialogue applications)
model is not directly accessible via an API today, but it powers
Bard – Google's answer to ChatGPT and is built on PaLM 2.
Cohere AI: Startup, which founders from the original
Transformer paper (Attention is all you need), has an LLM, but
also other products like Neural Search and Embed.
 Meta: Meta has recently released Llama 2 – a new family of
open-source AI language models that can be used in
commercial products. The models come in sizes ranging from 7
to 70 billion parameters and have been fine-tuned to
outperform most open-source chat models on benchmarks.
These are available to deploy via the Model Catalog in Azure
Machine Learning Studio, part of Azure AI.

2.5.2 Open Source LLMs

A new crop of OSS LLMs is coming up – some of these would end up
competing head-to-head with ChatGPT. As Figure 2.1 shows, there
are too many to mention, but a few noteworthy ones are listed
below in Table 2.2.

Table 2.2 Open Source LLMs

Company Open Source LLM Parameter

Size
Meta Llama LLM is one of the models that has Various
inspired many other OSS models. Llama (7B – 65
comes in many sizes (7B, 13B, 33B, and B)
65B), and while smaller than GPT3, it
can match it across many tasks. Meta
shared the models with researchers
(and was separately also leaked online).
This has inspired many others to use
that as a jumping-off point.
Databricks Databricks recently released v2 of Dolly, 12B
which they label "World's first truly open
Instruction-tuned LLM". This is being
released under the CCA 3.0 license,
allowing anyone to use, extend, and
modify this, including commercial
purposes.
Alpaca Standford University's Alpaca, an 7B
instruction model based on Llama,
claims to match GPT 3.5 Turbo
performance in some tasks.
FreedomGPT This is an OSS conversational agent Not
based on Alpaca. They claim to offer disclosed
100% uncensored and private
conversations.
Vicuna Academic researchers from multiple 13B
institutions (UC Berkeley, CMU,
Stanford, UC San Diego, and MBZUAI)
released Vicuna, a fine-tuned version of
Llama that matches GPT4 performance
across many tasks.
Koala Berkley AI Research released Koala, a 13B
fine-tuned version of Llama using
internet dialogues.
ChatLLaMa Technically, it's not a model but tooling 7B
for models. Nebuly AI released
ChatLLaMa, a library to create ChatGPT-
like conversational assistance using your
data.
ColossalChat UC Berkeley's ColossalAI project 7B
releases ColossalChat, a ChatGPT-like
model including complete RLHF
pipelines based on Llama.
Falcon The Technology Innovation Institute Various
(TII) in the United Arab Emirates (1B –
released a family of LLMs called the 180B)
Falcon LLM model. At the time, Falcon
was the biggest OSS LLM ever released
and was at the top of the OSS LLM
Leaderboard. More recently, a more
powerful 180B parameters model is
again at the top of the leaderboard.
OpenAI vs. Azure OpenAI

Azure OpenAI and OpenAI are both services that provide access to
OpenAI's powerful language models, but they have some key
differences. Generally speaking, OpenAI caters to more small-
medium-business individual developers and startups. Azure OpenAI,
on the other hand, is more for enterprises that need additional
security, availability in different parts of the world, and regulatory
needs.

Azure OpenAI offers additional enterprise-ready features, such as

data privacy, customer-managed keys, encryption at rest, private
networking, regional availability, and responsible AI content filtering.
These features can be important for businesses that need to comply
with specific security or regulatory requirements.

As a result, the APIs between the two are similar but not the same.
It is important to call out that the underlying models, however, are
the same, and Azure OpenAI has its deployment that incorporates
these additional features that most enterprises require.

2.6 Key Concepts of LLMs

This section covers the architecture of a typical LLM implementation.
Figure 2.3 below outlines at a high level the conceptual architecture
of a typical LLM implementation; whenever we call an LLM such as
OpenAI's GPT, it is the process it is going through.

This model starts with the input text – the prompt. The prompt is
first converted into a sequence of tokens using tokenization. Each
token is then converted into a numerical vector via a process called
embedding, which acts as the encoder input.

The encoder processes the input sequence and generates a

sequence of hidden states. These hidden states are then fed into the
decoder with a start token. The decoder generates the output
sequence one token at a time by predicting the next token based on
the previous tokens and hidden states.

Once all the layers have processed the information, the model
outputs a prediction for the net token in the learned sequence. This
outcome is converted back to the text and is the response we see.
This process runs in an iterative loop and occurs for each new token
generated, hence creating a coherent text output. The final text that
the model generates is an emergent property of this layered,
iterative process. The final output sequence is also called a
Completion.

Figure 2.3 Conceptual Architecture of an LLM

Let us examine each of these aspects of LLMs in more detail.

2.6.1 Prompts
A prompt is how we "talk" to these models. A prompt is just text
describing the task we are trying to achieve using natural language.
The output of these models is also text. The ability to express our
intention in this manner (natural language) instead of conforming to
the input restrictions of a machine makes prompts powerful. Crafting
or designing the text in the prompt is akin to programming the
model and creating a new paradigm called Prompt Engineering,
which we cover later in the book.

Prompts allow us to capture a broader set of intents and emotions

using the same paradigm. Below are two examples (generated by
AI) where a prompt asks for a Haiku outlining how great LLMs are –
one is happy, and the other is sad. Getting this level of emotion and
expressiveness on the same task was impossible.

Prompt: Write a happy haiku on how great LLMs are.

AI-Generated:

LLMs are great

They can chat and generate
Creative and smart
Prompt: Write a sad haiku on how great LLMs are.

AI-Generated:

LLMs are great

But they cannot feel or love
Only simulate

Note:

Large language models do not "understand" text like humans do.

They are statistical models that learn patterns in data and use those
patterns to generate output. Despite their impressive output, these
models don't comprehend the content generated in the same sense
as ours – including beliefs, desires, consciousness, and right and
wrong. They are just extremely good at predicting the next word in
a sequence of text based on patterns they've seen millions of times.

2.6.2 Tokens
Tokens are the basic units of text that a large language model (LLM)
uses to process both the request and the response, i.e., understand
and generate text. Tokenization is the process of converting text into
a sequence of smaller units called tokens. When using LLMs, we use
tokens to converse with these models, one of the most fundamental
elements to understanding LLMs.

Tokens are the new currency when incorporating LLMs into your
application or solutions. Tokens directly correlate with the cost of
running models – both in terms of money and cost in terms of the
experience with latency and throughput. The more tokens, the more
processing the model has to do. This means more computational
resources are required for the model, which means lower
performance and higher latency.

Tokens are a part of the text that LLMs process; this part can be
individual characters, words, sub-words, or even larger linguistic
units, depending on the tokenization algorithm. A rough rule of
thumb is that one token is approximately four characters or 0.75
words for English text. For most LLMs today, the token size that they
support includes both the input prompt and the response.

Let's see with an example; Figure 2.4 below shows how the
sentence "I have a white dog named Champ" gets tokenized (using
OpenAI's tokenizer in this case). Each block represents a different
token. In this example, we use eight tokens.

Figure 2.4 Tokenizer example

LLMs generate text by predicting the next word or symbol (token)
most likely to follow a given sequence of words or symbols (tokens)
they use as input, i.e., the prompt. We show a visual representation
of this in Figure 2.5 below, where the list of tokens on the right
shows the highest probability of tokens following the prompt “The
dog sat on”. We can influence some of this probability of tokens
using a few parameters we will see later in the chapter.

Figure 2.5 LLM – next token predictor

Suppose we have a sequence of tokens with a length of n. Utilizing

these n tokens as the context, we generate the subsequent token,
n+1. This newly predicted token is then appended to the original
sequence of tokens, thereby expanding the context. Consequently,
the expanded context window for generating token n+2 becomes n
+ (n+1). This process is repeated in a continuous loop until a
Another random document with
no related content on Scribd:
 Then Dode, his mother and sister began most unconcernedly to
speculate as to what if anything was next to be done with the old
farmer, the while the latter rolled a vacant eye over a scene he was
no longer able to interpret.
 X
MARRIAGE—FOR ONE

W HENEVER I think of love and marriage I think of Wray. That

clerkly figure. That clerkly mind. He was among the first people
I met when I came to New York and, like so many of the millions
seeking to make their way, he was busy about his affairs.
Fortunately, as I saw it, with the limitations of the average man he
had the ambitions of the average man. At that time he was
connected with one of those large commercial agencies which
inquire into the standing of business men, small and large, and
report their findings, for a price, to other business men. He was very
much interested in his work and seemed satisfied that should he
persist in it he was certain to achieve what was perhaps a fair
enough ambition: a managership in some branch of this great
concern, which same would pay him so much as five or six thousand
a year. The thing about him that interested me, apart from a genial
and pleasing disposition, was the fact that with all his wealth of
opportunity before him for studying the human mind, its resources
and resourcefulness, its inhibitions and liberations, its humor,
tragedy, and general shiftiness and changefulness, still he was
largely concerned with the bare facts of the differing enterprises
whose character he was supposed to investigate. Were they
solvent? Could and did they pay their bills? What was their capital
stock? How much cash did they have on hand?... Such was the
nature of the data he needed, and to this he largely confined himself.
Nevertheless, by turns he was amused or astonished or made
angry or self-righteous by the tricks, the secretiveness, the errors
and the downright meanness of spirit of so many he had to deal with.
As for himself, he had the feeling that he was honest,
straightforward, not as limited or worthless as some of these others,
and it was on this score that he was convinced he would succeed, as
he did eventually, within his limitations, of course. What interested
me and now makes me look upon him always as an excellent
illustration of the futility of the dream of exact or even suitable
rewards was his clerkly and highly respectable faith in the same. If a
man did as he should do, if he were industrious and honest and
saving and courteous and a few more of those many things we all
know we ought to be, then in that orderly nature of things which he
assumed to hold one must get along better than some others. What
—an honest, industrious, careful man not do better than one who
was none of these things—a person who flagrantly disregarded
them, say? What nonsense. It must be so. Of course there were
accidents and sickness, and men stole from one another, as he saw
illustrated in his daily round. And banks failed, and there were trusts
and combinations being formed that did not seem to be entirely in
tune with the interests of the average man. But even so, all things
considered, the average man, if he did as above, was likely to fare
much better than the one who did not. In short, there was such a
thing as approximate justice. Good did prevail, in the main, and the
wicked were punished, as they should be.
And in the matter of love and marriage he held definite views also.
Not that he was unduly narrow or was inclined to censure those
whose lives had not worked out as well as he hoped his own would,
but he thought there was a fine line of tact somewhere in this matter
of marriage which led to success there quite as the qualities outlined
above led, or should lead, to success in matters more material or
practical. One had to understand something about women. One had
to be sure that when one went a-courting one selected a woman of
sense as well as of charm, one who came of good stock and hence
would be possessed of good taste and good principles. She need
not be rich; she might even be poor. And one had to be reasonably
sure that one loved her. So many that went a-courting imagined they
loved and were loved when it was nothing more than a silly passing
passion. Wray knew. And so many women were designing, or at
least light and flighty; they could not help a serious man to succeed if
they would. However, in many out-of-the-way corners of the world
were the really sensible and worthy girls, whom it was an honor to
marry, and it was one of these that he was going to choose. Yet even
there it was necessary to exercise care: one might marry a girl who
was too narrow and conventional, one who would not understand the
world and hence be full of prejudices. He was for the intelligent and
practical and liberal girl, if he could find her, one who was his mental
equal.
It was when he had become secretary to a certain somebody that
he encountered in his office a girl who seemed to him to embody
nearly all of the virtues or qualities which he thought necessary. She
was the daughter of very modestly circumstanced parents who dwelt
in the nearby suburb of O——, and a very capable and faithful
stenographer, of course. If you had seen the small and respectable
suburb from which she emanated you would understand. She was
really pretty and appeared to be practical and sensible in many
ways, but still very much in leash to the instructions and orders and
tenets of her home and her church and her family circle, three worlds
as fixed and definite and worthy and respectable in her thought as
even the most enthusiastic of those who seek to maintain the order
and virtue of the world would have wished. According to him, as he
soon informed me—since we exchanged nearly all our affairs
whenever we met, she was opposed to the theatre, dancing, any
form of night dining or visiting in the city on weekdays, as well as
anything that in her religious and home world might be construed as
desecration of the Sabbath. I recall him describing her as narrow “as
yet,” but he hoped to make her more liberal in the course of time. He
also told me with some amusement and the air of a man of the world
that it was impossible for him to win her to so simple an outing as
rowing on the Sabbath on the little river near her home because it
was wrong; on the contrary, he had to go to church with her and her
parents. Although he belonged to no church and was mildly
interested in socialism, he kept these facts from her knowledge. The
theatre could not even be mentioned as a form of amusement and
she could not and would not dance; she looked upon his inclination
for the same as not only worldly but loose and sinful. However, as he
told me, he was very fond of her and was doing his best to influence
and enlighten her. She was too fine and intelligent a girl to stick to
such notions. She would come out of them.
 By very slow degrees (he was about his business of courting her
all of two or three years) he succeeded in bringing her to the place
where she did not object to staying downtown to dinner with him on a
weekday, even went with him to a sacred or musical concert of a
Sunday night, but all unbeknown to her parents or neighbors, of
course. But what he considered his greatest triumph was when he
succeeded in interesting her in books, especially bits of history and
philosophy that he thought very liberal and which no doubt
generated some thin wisps of doubt in her own mind. Also, because
he was intensely fond of the theatre and had always looked upon it
as the chiefest of the sources of his harmless entertainment, he
eventually induced her to attend one performance, and then another
and another. In short, he emancipated her in so far as he could, and
seemed to be delighted with the result.
With their marriage came a new form of life for both of them, but
more especially for her. They took a small apartment in New York, a
city upon which originally she had looked with no little suspicion, and
they began to pick up various friends. It was not long before she had
joined a literary club which was being formed in their vicinity, and
here she met a certain type of restless, pushing, seeking woman for
whom Wray did not care—a Mrs. Drake and a Mrs. Munshaw,
among others, who, from the first, as he afterward told me, he knew
could be of no possible value to any one. But Bessie liked them and
was about with them here, there, and everywhere.
It was about this time that I had my first suspicion of anything
untoward in their hitherto happy relations. I did not see him often
now, but when I did visit them at their small apartment, could not
help seeing that Mrs. Wray was proving almost too apt a pupil in the
realm in which he had interested her. It was plain that she had been
emancipated from quite all of her old notions as to the sinfulness of
the stage, and in regard to reading and living in general. Plainly,
Wray had proved the Prince Charming, who had entered the secret
garden and waked the sleeping princess to a world of things she had
never dreamed of. She had reached the place where she was
criticizing certain popular authors, spoke of a curiously enlightened
history of France she was reading, of certain bits of philosophy and
poetry which her new club were discussing. From the nature of the
conversation being carried on by the three of us I could see that
Wray was beginning to feel that the unsophisticated young girl he
had married a little while before might yet outstrip him in the very
realm in which he had hoped to be her permanent guide. More than
once, as I noticed, she chose to question or contradict him as to a
matter of fact, and I think he was astonished if not irritated by the fact
that she knew more than he about the import of a certain plot or the
relativity of certain dates in history. And with the force and
determination that had caused her to stand by her former
convictions, she now aired and defended her new knowledge. Not
that her manner was superior or irritating exactly; she had a friendly
way of including and consulting him in regard to many things which
indicated that as yet she had no thought of manifesting a superiority
which she did not feel. “That’s not right, dearest. His name is
Bentley. He is the author of a play that was here last year—The
Seven Rings of Manfred—don’t you remember?” And Wray, much
against his will, was compelled to confess that she was right.
Whenever he met me alone after this, however, he would confide
the growing nature of his doubts and perplexities. Bessie was no
more the girl she had been when he first met her than he was like
the boy he had been at ten years of age. A great, a very great
change was coming over her. She was becoming more aggressive
and argumentative and self-centred all the time, more this, more
that. She was reading a great deal, much too much for the kind of life
she was called upon to lead. Of late they had been having long and
unnecessary arguments that were of no consequence however they
were settled, and yet if they were not settled to suite her she was
angry or irritable. She was neglecting her home and running about
all the time with her new-found friends. She did not like the same
plays he did. He wanted a play that was light and amusing, whereas
she wanted one with some serious moral or intellectual twist to it.
She read only serious books now and was attending a course of
lectures, whereas he, as he now confessed, was more or less bored
by serious books. What was the good of them? They only stirred up
thoughts and emotions which were better left unstirred. And she liked
music, or was pretending she did, grand opera, recitals and that sort
of thing, whereas he was not much interested in music. Grand opera
bored him, and he was free to admit it, but if he would not
accompany her she would go with one or both of those two wretched
women he was beginning to detest. Their husbands had a little
money and gave them a free rein in the matter of their social and
artistic aspirations. They had no household duties to speak of and
could come and go as they chose, and Wray now insisted that it was
they who were aiding and abetting Bessie in these various interests
and enthusiasms and stirring her up to go and do and be. What was
he to do? No good could come if things went on as they were going.
They were having frequent quarrels, and more than once lately she
had threatened to leave him and do for herself here in New York, as
he well knew she could. He was doing very well now and they could
be happy together if only these others could be done away with.
It was only a month or two after this that Wray came to see me, in
a very distrait state of mind. After attempting to discuss several other
things quite casually he confessed that his young wife had left him.
She had taken a room somewhere and had resumed work as a
stenographer, and although he met her occasionally in the subway
she would have nothing to do with him. She wanted to end it all. And
would I believe it? She was accusing him of being narrow and
ignorant and stubborn and a number of other things! Only think of it!
And three or four years ago she had thought he was all wrong when
he wanted to go rowing on Sunday or stay downtown to dinner of an
evening. Could such things be possible? And yet he loved her, in
spite of all the things that had come up between them. He couldn’t
help it. He couldn’t help thinking how sweet and innocent and
strange she was when he first met her, how she loved her parents
and respected their wishes. And now see. “I wish to God,” he
suddenly exclaimed in the midst of the “oldtime” picture he was
painting of her, “that I hadn’t been so anxious to change her. She
was all right as she was, if I had only known it. She didn’t know
anything about these new-fangled things then, and I wasn’t satisfied
till I got her interested in them. And now see. She leaves me and
says I’m narrow and stubborn, that I’m trying to hold her back
intellectually. And all because I don’t want to do all the things she
wants to do and am not interested in the things that interest her,
now.”
I shook my head. Of what value was advice in such a situation as
this, especially from one who was satisfied that the mysteries of
temperament of either were not to be unraveled or adjusted save by
nature—the accidents of chance and affinity, or the deadly
opposition which keep apart those unsuited to each other?
Nevertheless, being appealed to for advice, I ventured a silly
suggestion, borrowed from another. He had said that if he could but
win her back he would be willing to modify the pointless opposition
and contention that had driven her away. She might go her
intellectual way as she chose if she would only come back. Seeing
him so tractable and so very wishful, I now suggested a thing that
had been done by another in a somewhat related situation. He was
to win her back by offering her such terms as she would accept, and
then, in order to bind her to him, he was to induce her to have a
child. That would capture her sympathy, very likely, as well as
insinuate an image of himself into her affectionate consideration.
Those who had children rarely separated—or so I said.
The thought appealed to him intensely. It satisfied his practical and
clerkly nature. He left me hopefully and I saw nothing of him for
several months, at the end of which time he came to report that all
was once more well with him. She had come back, and in order to
seal the new pact he had taken a larger apartment in a more
engaging part of the city. Bessie was going on with her club work,
and he was not opposing her in anything. And then within the year
came a child and there followed all those simple, homey, and
seemingly binding and restraining things which go with the rearing
and protection of a young life.
But even during that period, as I was now to learn, all was not as
smooth as I had hoped. Talking to me in Wray’s absence once
Bessie remarked that, delightful as it was to have a child of her own,
she could see herself as little other than a milch cow with an
attendant calf, bound to its service until it should be able to look after
itself. Another time she remarked that mothers were bond-servants,
that even though she adored her little girl she could not help seeing
what a chain and a weight a child was to one who had ambitions
beyond those of motherhood. But Wray, clerkly soul that he was, was
all but lost in rapture. There was a small park nearby, and here he
could be found trundling his infant in a handsome baby-carriage
whenever his duties would permit. He would sit or walk where were
others who had children of about the age of his own so that he might
compare them. He liked to speculate on the charm and innocence of
babyhood and was amused by a hundred things which he had never
noticed in the children of others. Already he was beginning to
formulate plans for little Janet’s future. It was hard for children to be
cooped up in an apartment house in the city. In a year or two, if he
could win Bessie to the idea, they would move to some suburban
town where Janet could have the country air.
They were prospering now and could engage a nursemaid, so
Mrs. Wray resumed her intellectual pursuits and her freedom.
Throughout it all one could see that, respect Wray as she might as a
dutiful and affectionate and methodical man, she could not love or
admire him, and that mainly because of the gap that lay between
them intellectually. Dissemble as he might, there was always the
hiatus that lies between those who think or dream a little and those
who aspire and dream much. Superiority of intellect was not
altogether the point; she was not so much superior as different, as I
saw it. Rather, they were two differing rates of motion, flowing side
by side for the time being only, his the slower, hers the quicker. And
it mattered not that his conformed more to the conventional thought
and emotions of the majority. Hers was the more satisfactory to
herself and constituted an urge which he feared rather than
despised; and his was more satisfactory to himself, compromise as
he would. Observing them together one could see how proud he was
of her and of his relationship to her, how he felt that he had captured
a prize, regardless of the conditions by which it was retained; and on
the other hand one could easily see how little she held him in her
thought and mood. She was forever talking to others about those
things which she knew did not interest him or to which he was
opposed.
 For surcease she plunged into those old activities that had so
troubled him at first, and now he complained that little Janet was
being neglected. She did not love her as she should or she could not
do as she was doing. And what was more and worse, she had now
taken to reading Freud and Kraft-Ebbing and allied thinkers and
authorities, men and works that he considered dreadful and
shameful, even though he scarcely grasped their true significance.
One day he came to me and said: “Do you know of a writer by the
name of Pierre Loti?”
“Yes,” I replied, “I know his works. What about him?”
“What do you think of him?”
“As a writer? Why, I respect him very much. Why?”
“Oh, I know, from an intellectual point of view, as a fine writer,
maybe. But what do you think of his views of life—of his books as
books to be read by the mother of a little girl?”
“Wray,” I replied, “I can’t enter upon a discussion of any man’s
works upon purely moral grounds. He might be good for some
mothers and evil for others. Those who are to be injured by a picture
of life must be injured, or kept from its contaminating influence, and
those who are to be benefited will be benefited. I can’t discuss either
books or life in any other way. I see worthwhile books as truthful
representations of life in some form, nothing more. And it would be
unfair to any one who stood in intellectual need to be restrained from
that which might prove of advantage to him. I speak only for myself,
however.”
It was not long after that I learned there had been a new quarrel
and that Bessie had left him once more, this time, as it proved, for
good. And with her, which was perhaps illegal or unfair, she had
taken the child. I did not know what had brought about this latest
rupture but assumed that it was due to steadily diverging views.
They could not agree on that better understanding of life which at
one time he was so anxious for her to have—his understanding. Now
that she had gone beyond that, and her method of going was
unsatisfactory to him, they could not agree, of course.
 Not hearing from him for a time I called and found him living in the
same large apartment they had taken. Its equipment was better
suited to four than to one, yet after seven or eight months of absence
on her part here he was, living alone, where every single thing must
remind him of her and Janet. As for himself, apart from a solemnity
and reserve which sprang from a wounded and disgruntled spirit, he
pretended an indifference and a satisfaction with his present state
which did not square with his past love for her. She had gone, yes;
but she had made a mistake and would find it out. Life wasn’t as she
thought it was. She had gone with another man—he was sure of
that, although he did not know who the man was. It was all due to
one of those two women she had taken up with, that Mrs. Drake.
They were always interested in things which did not and could not
interest him. After a time he added that he had been to see her
parents. I could not guess why, unless it was because he was lonely
and still very much in love and thought they might help him to
understand the very troublesome problem that was before him.
It was a year and a half before I saw him again, during which time,
as I knew, he continued to live in the apartment they had occupied
together. He had become manager of a department of the agency by
this time and was going methodically to and fro between his home
and office. After living alone and brooding for more than a year, he
came to see me one rainy November night. He looked well enough
materially, quite the careful person who takes care of his clothes, but
thinner, more tense and restless. He seated himself before my fire
and declared that he was doing very well and was thinking of taking
a long vacation to visit some friends in the West. (He had once told
me that he had heard that Bessie had gone to California.) Yes, he
was still living in the old place. I might think it strange, but he had not
thought it worth while to move. He would only have to find another
place to live in; the furniture was hard to pack; he didn’t like hotels.
Then of a sudden, noting that I studied him and wondered, he
grew restless and finally stood up, then walked about looking at
some paintings and examining a shelf of books. His manner was that
of one who is perplexed and undetermined, of one who has stood
out against a silence and loneliness of which he was intensely
weary. Then of a sudden he wheeled and faced me: “I can’t stand it.
That’s what’s the matter. I just can’t stand it any longer. I’ve tried and
tried. I thought the child would make things work out all right, but she
didn’t. She didn’t want a child and never forgave me for persuading
her to have Janet. And then that literary craze—that was really my
own fault, though. I was the one that encouraged her to read and go
to theatres. I used to tell her she wasn’t up-to-date, that she ought to
wake up and find out what was going on in the world, that she ought
to get in with intelligent people. But it wasn’t that either. If she had
been the right sort of woman she couldn’t have done as she did.” He
paused and clenched his hands nervously and dramatically. It was
as though he were denouncing her to her face instead of to me.
“Now, Wray,” I interposed, “how useless to say that. Which of us is
as he should be? Why will you talk so?”
“But let me tell you what she did,” he went on fiercely. “You haven’t
an idea of what I’ve been through, not an idea. She tried to poison
me once so as to get rid of me.” And here followed a brief and sad
recital of the twists and turns and desperation of one who was
intensely desirous of being free of one who was as desirous of
holding her. And then he added: “And she was in love with another
man, only I could never find out who he was.” And his voice fell to a
low, soft level, as though he was even then trying to solve the
mystery of who it was. “And I know she had an operation performed,
though I could never prove it.” And he gave me details of certain
mysterious goings to and fro, of secret pursuits on his part, actions
and evidences and moods and quarrels that pointed all too plainly to
a breach that could never be healed. “And what’s more,” he
exclaimed at last, “she tortured me. You’ll never know. You couldn’t.
But I loved her.... And I love her now.” Once more the tensely
gripped fingers, the white face, the flash of haunted eyes.
“One afternoon I stood outside of a window of an apartment house
when I knew she was inside, and I knew the name of the man who
was supposed to occupy it, only he had re-sublet it, as I found out
afterwards. And she had Janet with her—think of that!—our own little
girl! I saw her come to the window once to look out—I actually saw
her in another man’s rooms. I ran up and hammered at the door—I
tried to break it open. I called to her to come out but she wouldn’t,
and I went to get a policeman to make him open the door. But when I
got back a servant was coming up as though she had been out, and
she unlocked the door and went in. It was all a ruse, and I know it.
They weren’t inside. She had slipped out with Janet. And she had
told me they were going to Westchester for the day.
“And another time I followed her to a restaurant when she said she
was going to visit a friend. I suspected there was a man—the man I
thought she was going with, but it was some one I had never seen
before. When they came out and were getting into a cab I came up
and told them both what I thought of them. I threatened to kill them
both. And she told him to go and then came home with me, but I
couldn’t do anything with her. She wouldn’t talk to me. All she would
say was that if I didn’t like the way she was doing I could let her go.
She wanted me to give her a divorce. And I couldn’t let her go, even
if I had wanted to. I loved her too much. And I love her too much
now. I do. I can’t help it.” He paused. The pain and regret were
moving.
“Another time,” he went on, “I followed her to a hotel—yes, to a
hotel. But when I got inside she was waiting for me; she had seen
me. I even saw a man coming toward her—but not the one I believed
was the one—only when he saw me he turned away and I couldn’t
be sure that he was there to meet her. And when I tried to talk to her
about him she turned away from me and we went back home in
silence. I couldn’t do anything with her. She would sit and read and
ignore me for days—days, I tell you—without ever a word.”
“Yes,” I said, “but the folly of all that. The uselessness, the
hopelessness. How could you?”
“I know, I know,” he exclaimed, “but I couldn’t help it. I can’t now. I
love her. I can’t help that, can I? I’m miserable without her. I see the
folly of it all, but I’m crazy about her. The more she disliked me the
more I loved her. And I love her now, this minute. I can’t help it.
There were days when she tortured me so that I vomited, from sheer
nervousness. I was sick and run down. I have been cold with sweat
in her presence and when she was away and I didn’t know where
she was. I have walked the streets for hours, for whole days at a
time, because I couldn’t eat or sleep and didn’t know what to do. By
God!” Once more the pause and a clenching of the hands. “And all I
could do was think and think and think. And that is all I do now really
—think and think and think. I’ve never been myself since she went
away. I can’t shake it off. I live up there, yes. But why? Because I
think she might come back some day, and because we lived there
together. I wait and wait. I know it’s foolish, but still I wait. Why? God
only knows. And yet I wait. Oh,” he sighed, “and it’s three years now.
Three years!”
He paused and gazed at me and I at him, shaken by a fact that
was without solution by any one. Here he was—the one who had
understood so much about women. But where was she, the one he
had sought to enlighten, to make more up-to-date and liberal? I
wondered where she was, whether she ever thought of him even,
whether she was happy in her new freedom. And then, without more
ado, he slipped on his raincoat, took up his umbrella, and stalked out
into the rain, to walk and think, I presume. And I, closing the door on
him, studied the walls, wondering. The despair, the passion, the
rage, the hopelessness, the love. “Truly,” I thought, “this is love, for
one at least. And this is marriage, for one at least. He is spiritually
wedded to that woman, who despises him, and she may be
spiritually wedded to another man who may despise her. But love
and marriage, for one, at least, I have seen here in this room to-
night, and with mine own eyes.”
 XI
FULFILMENT

H EARING the maid tap lightly on her door for the third or fourth
time, Ulrica uttered a semiconscious “Come.” It was her usual
rising hour but to-day she was more depressed than usual, although
the condition was common enough at all times. The heavy drag of a
troubled mental state was upon her. Was it never to end? Was she
never to be happy again? After several weeks of a decidedly
acceptable loneliness, during which Harry had been in the west
looking after his interminable interests, he was about to return. The
weariness of that, to begin with! And while she could not say that she
really hated or even disliked him deeply (he was too kind and
considerate for that), still his existence, his able and different
personality, constantly forced or persuaded upon her, had come to
be a bore. The trouble was that she did not truly love him and never
could. He might be, as he was, rich, resourceful and generous to a
fault in her case, a man whom the world of commerce respected, but
how did that avail her? He was not her kind of man. Vivian before
him had proved that. And other men had been and would be as glad
to do as much if not more.
Vivian had given all of himself in a different way. Only Harry’s
seeking, begging eyes pleading with her (after Vivian’s death and
when she was so depressed) had preyed upon and finally moved her
to sympathy. Life had not mattered then, (only her mother and
sister), and she had become too weary to pursue any career, even
for them. So Harry with his wealth and anxiety to do for her—
(The maid entered softly, drew back the curtains and
raised the blinds, letting in a flood of sunshine, then
proceeded to arrange the bath.)
It had been, of course, because of the magic of her beauty—how
well she knew the magic of that!—plus an understanding and
sympathy she had for the miseries Harry had endured in his youth,
that had caused him to pursue her with all the pathetic vehemence of
a man of fifty. He was not at all like Vivian, who had been shy and
retiring. Life had seemed to frighten poor Vivian and drive him in
upon himself in an uncomplaining and dignified way. In Harry’s case
it had acted contrariwise. Some men were so, especially the old and
rich, those from whom life was slipping away and for whom youth,
their lost youth, seemed to remain a colored and enthralling
spectacle however wholly gone. The gifts he had lavished upon her,
the cars, the jewels, this apartment, stocks and bonds, even that
house in Seadale for her sister and mother! And all because of a
beauty that meant so little to her now that Vivian was gone, and in
the face of an indifference so marked that it might well have wearied
any man.
How could she go on? (She paused in her thoughts to survey and
follow her maid, who was calling for the second time.) Though he
hung upon her least word or wish and was content to see her at her
pleasure, to run her errands and be ever deferential and worshipful,
still she could not like him, could barely tolerate him. Before her
always now was Vivian with his brooding eyes and elusive, sensitive
smile; Vivian, who had never a penny to bless himself with. She
could see him now striding to and fro in his bare studio, a brush in
one hand, or sitting in his crippled chair meditating before a picture
or talking to her of ways and means which might be employed to
better their state. The pathos!
“I cannot endure that perfume, Olga!”
In part she could understand her acceptance of Harry after Vivian
(only it did not seem understandable always, even to her), for in her
extreme youth her parents had been so very poor. Perhaps because
of her longings and childish fears in those days she had been
marked in some strange way that had eventually led her to the
conviction that wealth was so essential. For her parents were
certainly harassed from her sixth to her thirteenth years, when they
recovered themselves in part. Some bank or concern had failed and
they had been thrown on inadequate resources and made to shift
along in strange ways. She could remember an old brick house with
a funereal air and a weedy garden into which they had moved and
where for a long time they were almost without food. Her mother had
cried more than once as she sat by the open window looking
desolately out, while Ulrica, not quite comprehending what it was all
about, had stared at her from an adjacent corner.
“Will madame have the iris or the Japanese lilac in the
water?”
She recalled going downtown once on an errand and slipping
along shyly because her clothes were not good. And when she saw
some schoolgirls approaching, hid behind a tree so they should not
see her. Another time, passing the Pilkington at dinner-time, the
windows being open and the diners visible, she had wondered what
great persons they must be to be able to bask in so great a world. It
was then perhaps that she had developed the obsession for wealth
which had led to this. If only she could have seen herself as she now
was she would not have longed so. (She paused, looking gloomily
back into the past.) And then had come the recovery of her father in
some way or other. He had managed to get an interest in a small
stove factory and they were no longer so poor—but that was after
her youth had been spoiled, her mind marked in this way.
And to crown it all, at seventeen had come Byram the inefficient.
And because he was “cute” and had a suggestion of a lisp; was of
good family and really insane over her, as nearly every youth was
once she had turned fourteen, she had married him, against her
parents’ wishes, running away with him and lying about her age, as
did he about his. And then had come trying times. Byram was no
money-maker, as she might have known. He was inexperienced, and
being in disfavor with his parents for ignoring them in his hasty
choice of a wife, he was left to his own devices. For two whole years
what had she not endured—petty wants which she had concealed
from her mother, furniture bought on time and dunned for, collectors
with whom she had to plead not to take the stove or the lamp or the
parlor table, and grocery stores and laundries and meat-markets
which had to be avoided because of unpaid bills. There had even
been an ejectment for non-payment of rent, and job after job lost for
one reason and another, until the whole experiment had been
discolored and made impossible even after comfort had been
restored.
“I cannot endure the cries of the children, Olga. You will
have to close that window.”
No; Byram was no money-maker, not even after his parents in far-
distant St. Paul had begun to help him to do better. And anyhow by
then, because she had had time to sense how weak he was, what a
child, she was weary of him, although he was not entirely to blame. It
was life. And besides, during all that time there had been the most
urgent pursuit of her by other men, men of the world and of means,
who had tried to influence her with the thought of how easily her life
could be made more agreeable. Why remain faithful to so young and
poor a man when so much could be done for her. But she had
refused. Despite Byram’s lacks she had small interest in them,
although their money and skill had succeeded in debasing Byram in
her young and untrained imagination, making him seem even
weaker and more ridiculous than he was. But that was all so long
ago now and Vivian had proved so much more important in her life.
While even now she was sorry for Harry and for Byram she could
only think of Vivian, who was irretrievably gone. Byram was
successful now and out of her life, but maybe if life had not been so
unkind and they so foolish——
“You may have Henry serve breakfast and call the car!”
And then after Byram had come Newton, big, successful,
important, a quondam employer of Byram, who had met her on the
street one day when she was looking for work, just when she had
begun to sense how inefficient Byram really was, and he had proved
kind without becoming obnoxious or demanding. While declaring,
and actually proving, that he wished nothing more of her than her
good-will, he had aided her with work, an opportunity to make her
own way. All men were not selfish. He had been the vice-president of
the Dickerson Company and had made a place for her in his office,
saying that what she did not know he would teach her since he
needed a little sunshine there. And all the while her interest in Byram
was waning, so much so that she had persuaded him to seek work
elsewhere so that she might be rid of him, and then she had gone
home to live with her mother. And Newton would have married her if
she had cared, but so grieved was she by the outcome of her first
love and marriage that she would not.
“The sedan, yes. And I will take my furs.”
And then, living with her mother and making her own way, she had
been sought by others. But there had been taking root and growing
in her an ideal which somehow in the course of time had completely
mastered her and would not even let her think of anything else, save
in moments of loneliness and the natural human yearning for life.
This somehow concerned some one man, not any one she knew, not
any one she was sure she would ever meet, but one so wonderful
and ideal that for her there could be no other like him. He was not to
be as young or unsophisticated as Byram, nor as old and practical
as Newton, though possibly as able (though somehow this did not
matter), but wise and delicate, a spirit-mate, some such wondrous
thing as a great musician or artist might be, yet to whom in spite of
his greatness she was to be all in all. She could not have told herself
then how she was to have appealed to him, unless somehow surely,
because of her great desire for him, her beauty and his
understanding of her need. He was to have a fineness of mind and
body, a breadth, a grasp, a tenderness of soul such as she had not
seen except in pictures and dreams. And such as would need her.
“To Thorne and Company’s first, Fred.”
Somewhere she had seen pictures of Lord Byron, of Shelley, Liszt
and Keats, and her soul had yearned over each, the beauty of their
faces, the record of their dreams and seekings, their something
above the common seeking and clayiness (she understood that
now). They were of a world so far above hers. But before Vivian
appeared, how long a journey! Life had never been in any hurry for
her. She had gone on working and seeking and dreaming, the while
other men had come and gone. There had been, for instance, Joyce
with whom, had she been able to tolerate him, she might have found
a life of comfort in so far as material things went. He was, however,
too thin or limited spiritually to interest a stirring mind such as hers, a
material man, and yet he had along with his financial capacity more
humanity than most, a kind of spiritual tenderness and generosity at
times towards some temperaments. But no art, no true romance. He
was a plunger in real estate, a developer of tracts. And he lacked
that stability and worth of temperament which even then she was
beginning to sense as needful to her, whether art was present or not.
He was handsomer than Byram, a gallant of sorts, active and
ebullient, and always he seemed to sense, as might a homing
pigeon, the direction in which lay his own best financial opportunities
and to be able to wing in that direction. But beyond that, what? He
was not brilliant mentally, merely a clever “mixer” and maker of
money, and she was a little weary of men who could think only in
terms of money. How thin some clever men really were!
“I rather like that. I’ll try it on.”
And so it had been with him as it had been with Byram and
Newton, although he sought her eagerly enough! and so it was
afterward with Edward and Young. They were all worthy men in their
way. No doubt some women would be or already had been drawn to
them and now thought them wonderful. Even if she could have
married any one of them it would only have been to have endured a
variation of what she had endured with Byram; with them it would
have been of the mind instead of the purse, which would have been
worse. For poor Byram, inefficient and inexperienced as he was, had
had some little imagination and longings above the commonplace.
But these, as contrasted with her new ideal——
“Yes, the lines of this side are not bad.”
Yes, in those days there had come to her this nameless unrest,
this seeking for something better than anything she had yet known
and which later, without rhyme or reason, had caused her to be so
violently drawn to Vivian. Why had Vivian always grieved so over her
earlier affairs? They were nothing, and she regretted them once she
knew him.
“Yes, you may send me this one, and the little one with
the jade pins.”