Introduction to Cloud Computing:

What is Cloud Computing?

Cloud Computing is a way of using IT Services with the following 5 traits:

 On Demand Self Service: Cloud-computing customers use an automated interface and get
the processing power, storage, and network they need, with no need for human
intervention.

 Broad Network Access: Resources can be accessed anywhere over the network.

 Resource Pooling: Provider of those resources has a big pool of them and allocates them to
customers out of the pool. That allows the provider to get economies of scale by buying in
bulk. Customers don’t have to know or care about the exact physical location of those
resources

 Rapid Elasticity: Customers who need more resources can get more rapidly. When they need
less, they can scale back

 Measured Service: the customers pay only for what they use or reserve, as they go. If they
stop using resources, they stop paying.

Google Cloud Platform offers four main kinds of services:

 Compute
 Storage
 Big Data
 Machine Learning

History: How did we get here and Where are we going?

 Physical/Colocation: User Configured, User Managed and User Maintained
Colocation gave users the financial efficiency of renting physical space, instead of investing
in data centre real estate
 Virtualized: User Configured, Provider-Managed and Provider-Maintained
The components of virtualized data centres match the physical building blocks of hosted
computing—servers, CPUs, disks, load balancers, and so on—but now they are virtual
devices.
Virtualization does provide several benefits: your development teams can move faster, and
you can turn capital expenses into operating expenses. With virtualization you still maintain
the infrastructure; it is still a user-controlled/user-configured environment
 Serverless: Fully Automated
About 10 years ago, Google realized that its business couldn’t move fast enough within the
confines of the virtualization model. So, Google switched to a container-based
architecture—a fully automated, that consists of a combination of automated services and
scalable data. Services automatically provision and configure the infrastructure used to run
applications.

Every Company is a Data Company:

Every company—regardless of size or industry—will differentiate itself from its competitors through
technology. Largely, that technology will be in the form of software. Great software is centred on
data. Thus, every company is or will become a data company
GCP Computing Architectures:
 Compute Engine (IaaS) Towards Managed Infrastructure
 Kubernetes Engine (Hybrid)
 App Engine (PaaS)
 Cloud Functions (Serverless logic)
 Managed Services (Automated elastic resources) Towards Dynamic Infrastructure

IaaS offerings provide you with raw compute, storage, and network, organized in ways familiar to
you from physical and virtualized data centres. Where you pay for what you allocate.

PaaS offerings, on the other hand, bind your code to libraries that provide access to the
infrastructure your application needs, thus allow you to focus on your application logic. Where you
pay for what you use.

Benefits from GCP:

 Billing in Sub-hour Increments: For Compute, data processing and other services
 Discounts for sustained use: Automatically applied to virtual machine use over 25% of a
month
 Discounts for committed use: Pay less for steady, long term workloads
 Discounts for preemptible use: Pay less for interruptible workloads
 Custom VM instance type: pay only for the resources you need for your application

Google gives customers the ability to run their applications elsewhere if Google becomes no longer
the best provider for their needs. Which includes:

 Using Open APIs. Google services are compatible with open-source products
 Google publishes key elements of its technology, using open-source licenses, to create
ecosystems that provide customers with options other than Google
 Google provides interoperability at multiple layers of the stack. Kubernetes and Google
Kubernetes Engine give customers the ability to mix and match microservices running across
different clouds. Google Stack driver lets customers monitor workloads across multiple cloud
providers

Google Cloud Platform lets you choose from computing, storage, big data/machine learning, and
application services for your web, mobile, analytics, and backend solutions.
Resource Manager and Hierarchy:
Virtual Machines in the Cloud:
 Compute engine lets us run the VM in the Google Cloud Infrastructure
 Each VM has a power of a full-fledged operating system and can configure them like we
build a physical server by specifying CPU power and memory, amount and type of storage,
operating system.
 VMs needs a physical storage and can choose between two kinds
o Persistent storage/disks – Stored data is not lost when VM terminates
o SSD – Local SSD gives high performance scratch space, but data stored gets lost
when the VM terminates.
 GCP offers Linux and Windows ready to go versions, also there is an option of loading own
images as well.
 In GCP VM we can include start up scripts that can install software packages on first boot
etc. We can even take a durable snapshot of the disks from the running VM and can use
them as backup when migrating VM to another region.
 A preemptible VM is different from ordinary compute engine VM in one aspect, where
compute engine is given permission to terminate if the bash jobs completes and if its
resources are needed elsewhere. This can save money as well.
 Using customer build VMs we can create “huge VMs – which are great for in memory DB and
CPU intensive analytics”, Also Compute engines has the feature called Auto scaling which
adds and removes VMs based upon work load and balancing the incoming traffic across
VMs.

Virtual Private Cloud (VPC) Network:

 VPC Networks connect GCP resources to each other and to the internet, where the network
can be segmented and firewall rules to restrict access to instances. we can create static
routes to forward traffic to specific destinations.
 VPC network has global scope and can have subnets in any GCP region worldwide.
 VPC has below built in features
o Routing tables – use to forward traffic from one instance to another instance
o Firewall instance – Control and restrict access to instances both incoming and
outgoing traffic.
 VPCs belongs to GCP projects, where we can establish peering relationship between two
VPCs (Projects) so that they can exchange traffic.
 Cloud Load Balancer is a fully distributed software defined managed service for all traffic.
Where Cloud Load Balancer don’t run in VMs and we have to put the Load Balancer in front
of traffic.
o HTTPS – Cross regional load balancing for web applications
o SSL proxy – For Secure socket layer traffic that is not HTTP (Supported on Specific
ports)
o TCP Proxy – For TCP traffic that does not use Secure Socket (Supported on Specific
ports)
o Regional - balance UDP traffic or traffic on any port number, you can still load
balance across a GCP region with the regional load balancer.

Above 4 are for the traffic that is coming into Google network from the internet.
 o Internal – to Load Balance the traffic inside any project, like between the
presentation layer and business logic layer.

GCP Cloud Storage:

Apart from the VMs persistent disks, there are other storage options:

 Cloud Storage (Object Storage) –

o Not a File storage, Managed in hierarchy of folders
o Not a Block storage, Where OS manages data as chunks of disk
o It comprises of Buckets, where we create and configure to hold the bunch of
bytes/data given and address them with a unique key (form of URLs)
o The Storage objects are immutable, which means that we cannot edit them but
instead create new versions.
o Cloud Storage always encrypts data on the server side at no extra cost and by
default data-transit is encrypted using HTTPS.
o Large amounts of data can be moved to Cloud Storage and from there data can be
moved to other GCP storage services.
o When you create a cloud bucket, provide a globally unique name, geographic
location and storage class. There are 4 different storage classes:

Multi-Regional (High Regional (High Nearline (Backup and Cold line (Backup and
Performance) Performance) Archival) Archival)
Most frequently Accessed Frequently Accessed less than Accessed less than
accessed data. within a region. once in a month. once in a year.
Very High Price per GB High Price per GB Low Price per GB Very low price per GB
Stored per month. Stored per month. Stored per month. Stored per month.
Very low Price per GB low Price per GB High Price per GB Very high Price per GB
transferred. transferred. transferred. transferred.
Example: website Example: to store data In scenarios where For data archiving,
content, interactive close to their compute you plan to read or online backup, and
workloads, or data engine, virtual modify your data once disaster recovery
that's part of mobile machines, or their a month or less on
and gaining Kubernetes engine average.
applications clusters

o Object versioning is optional, where if we switch on then we can have list of

archived versions and if it is off then always the new version will replace the old.
o GCP offers life-cycle management policies to control how many versions can be
archived when the object versioning is switched on.
o In short, cloud storage is often the ingestion point for data being moved into the
cloud and is frequently the long-term storage location for data
o Overall it is best for structured and unstructured, binary or object data like images,
large media files and backups.
 Cloud SQL –
o It is a RDBMS and offers MySQL
o Offers a choice between MySQL and PostgreSQL databases which are capable if
handling TBs of data.
 o if an outage occurs, Cloud SQL can replicate data between multiple zones with
automatic failover. Cloud SQL also helps backup your data with either On-Demand
or scheduled backups. It can also scale both vertically by changing the machine type,
and horizontally via read replicas.
o Authorize compute engine instances for access Cloud SQL instances and configure
the Cloud SQL instance to be in the same zone as your virtual machine. Cloud SQL
also supports other applications and tools that you might be used to like SQL Work
Bench, Toad, and other external applications using standard MySQL drivers.
o It is best for web frameworks and in existing applications like customer orders.
 Cloud Spanner –
o It provides horizontal scale ability.
o It offers transactional consistency at a global scale, schemas, SQL, and automatic
synchronous replication for high availability and it can provide petabytes of capacity.
o Consider using Cloud Spanner if you have outgrown any relational database, or
sharing your databases for throughput high performance, need transactional
consistency, global data and strong consistency, or just want to consolidate your
database. Natural use cases include, financial applications, and inventory
applications.
o It is best for Large scale DB applications that are larger than two terabytes, for
example for financial trading and e-commerce use cases.
 Cloud Data Store –
o It is Highly scalable No SQL Database and stores structured data from App Engines
applications.
o Automatically handles sharing and replication, providing you with a highly available
and durable database that scales automatically to handle load.
o Unlike Cloud Bigtable, it also offers transactions that affect multiple database rows,
and it lets you do SQL-like queries.
o Cloud Datastore has a free daily quota that provides storage, reads, writes, deletes
and small operations at no charge.
o Overall it is best for semi structured application data that is used in app engines
applications.
 Google Big Table –
o It is Google’s No SQL, Big Database Service.
o It is ideal for storing large amounts of data with very low latency, hence it is a great
choice for both operational and analytical applications including IoT, User Analytics
and financial data analysis.
o Cloud Big Table is offered through the same open source API as HBase, which is a
native DB for Apache Hadoop project.
o Overall it is best for analytical data with heavy read and write operations.
 Cloud Big Table Cloud Storage Cloud SQL Cloud
Datastore Spanner
Type No SQL No SQL wide Blod store Relational Relational
Document column SQL SQL
(Unstructured) (Structured)
Support for Yes No and Single No Yes Yes
SQL Row
Transactions transactions
Complex No No No Yes Yes
queries
Capacity TB+ PB+ PB+ TB+ PB+

Containers and Kubernetes:

IaaS (Compute Engine) < ------------ > Kubernetes/Containers < ------------- > PaaS (App Engine)

IaaS:

 It allows to share the computing resources by virtualizing the hardware using virtual
machines.
 Here one must install web server, database, middleware, configure the underlying system
resources such as disk space, disk IO or networking. However, as demand for the application
increases, we need to copy the an entire VM and install OS for each instance of the app
which can be slow and costly as well.

PaaS:

 With App engine we get access to programming services, so all we can do is write the code
and self-contained workloads that uses these services and include any dependent libraries.
 As Demand for the application increases, the platform scales your app seamlessly and
independently by workload and infrastructure. But the underlying infrastructure cannot be
finetuned to save cost.

Kubernetes/Containers:

 The idea of container is to give independent scalability of workload and an abstraction layer
of OS and Hardware.
 All you need on each host is an OS kernel that supports containers, and a container runtime.
You’re virtualizing the OS, it scales like paths, but gives you nearly the same flexibility as
IAAS. With this abstraction, your code is ultra-portable, and you can treat the OS and
hardware as a black box.
 'll use an open source tool called Docker, that defines a format for bundling your application,
its dependencies, and machine specific settings into a container. You could use a different
tool like Google Container Builder.
 Kubernetes is tool that helps in orchestrate many containers on many hosts, scale them as
microservices, and deploy rollouts and rollbacks.
 Kubernetes is an open source orchestrator that abstracts containers at a higher level so you
can better manage and scale your applications. At the highest level, Kubernetes is a set of
APIs that you can use to deploy containers on a set of nodes called a cluster. The system is
 divided into a set of master components that run as a control plane, and a set of nodes that
run containers.

App Engine:
The App engine (PaaS) platform manages the hardware and networking infrastructure required to
run the code. To Deploy an application in App engine, we just need to deploy app engineer code and
the App engine service takes care of the rest. App Engine provides built in services that many web
applications need like No SQL DB, in memory caching, load balancing, health checks, logging and
authenticate users. App engine will scale the application in response to the amount of traffic it
receives. App Engine is suited for application where the workload is highly variable or unpredictable
like web application and mobile backend. It offers to 2 environments: Standard and flexible.

App Engine Standard environment:

 Fine Grain Autoscaling
 Free daily usage Quota for some services (Low utilization applications might run at no
charge)
 Provides runtime libraries for specific versions of java, phyton, PHP and Go. The Runtime
libraries also include support App engine APIs.
 The Standard environment also enforces restrictions on your code by making them run in
sandbox. A sandbox is a software construct that’s independent of the hardware, OS or
physical location of the server it runs on. The Sandbox is one of the reasons why App Engine
Standard Environment can scale and manage application in a very fine-grained way. Below
are the restrictions of using sandbox:
o Cannot write to local file system
o Request application receives has a 60 second timeout
o Cannot install arbitrary 3rd party software

App Engine Flexible environment:

 App engine flexible runs inside Docker containers on Compute engine VMs. App Engine
manages these compute engines.
 No Sandbox Restrictions
 Can Access App engine resources
 Instance start is little slow than standard
 App Engine Flexible is between Kubernetes and App Engine Standard

GCP offers 2 API Management tools:

 Cloud Endpoints: developing a software service and one of GCP's backends, you'd like to
make it easy to expose this API, you'd like to make sure it's only consumed by other
developers whom you trust, you'd like an easy way to monitor and log its use, you'd like for
the API to have a single coherent way for it to know which end user is making the call. That's
when you use Cloud Endpoints.
 Apigee Edge: Focus on Business problems like rate limiting, quotas and analytics. Many
users of Apigee Edge are providing a software service to other companies and those
features come in handy. Because of the backend services for Apigee Edge need not be in
GCP, engineers often use it when they are taking apart a legacy application.
Cloud Source repositories:
It provides GIT version control to keep code private to a GCP project and use IAM permissions to
protect but not to maintain the Git repositories. Using this we can have any number of private Git
repositories. It also provides source viewer so that we can view and browse repository files within
GCP console.

Cloud Functions:
Cloud functions can trigger on events in the cloud storage, cloud pub/sub or in a HTTP call. You
choose which events you care about. For each event type, you tell cloud functions you're interested
in it. These declarations are called triggers. Then you attach JavaScript functions to your triggers.
From now on, your functions will respond whenever the events happen.

Deployment Manager: Infrastructure as a code

It is an infrastructure manager service that automates the creation and management of your GCP
resources. you can create a template file using either the YAML mark-up language or Python that
describes what you want the components of your environment to look like. Then, you give the
template to deployment manager which figures out and does the actions needed to create the
environment your template describes. If you need to change your environment, edit your template
and then tell Deployment Manager to update the environment to match the change. we can store
and version control your Deployment Manager templates in Cloud Source repositories.

Stackdriver: Monitoring

It offers capabilities in five areas:

 Monitoring – Platform, system and application metrics, health checks, Dashboards and alerts
 Logging – Platform, system and application logs: log search, view, filter and export
 Trace – Latency reporting and sampling, per-URL latency and statistics
 Debugger - Debug Applications
 Error Monitoring – Error Notifications and Error Dashboard

Google Cloud Big Data Services: