Elastic Cloud Storage

Lakmal Warusawithana

Dilan Gallage
Associate Lead Systems Engineer - DevOps, WSO2 Inc
2019 - SLIIT - Cloud Computing
dilang@wso2.com
Introduction to Elastic Cloud
Storage

Lakmal Warusawithana
Elastic Cloud Storage
● Elasticity in cloud storage is the ability for the
organization to adjust its storage requirements in
terms of capacity and processing with respect to
operational requirements.
● Typical Cloud Storage provide by using block storage.
● Block storage is a type of data storage typically used
in storage-area network (SAN) environments where
data is stored in volumes, also referred to as blocks.
Lakmal Warusawithana

Introduction to Elastic Cloud

Storage
Need of Elastic Cloud Storage
● Why people need storage on the cloud?
- Easy to store as much as data you want and
access it when you need it
- Avoid having to buy hardware and paying for
storage that isn’t be used
- It can be used to store and retrieve any amount of
data, any time, from anywhere on the web.
- Only have to pay for storage and bandwidth you
use Lakmal Warusawithana
- Can be used as backup and storage, application or
media hosting, hosting high traffic website etc.
Introduction to Elastic Cloud
Storage
Elastic Cloud Storage - Benefits
● Operational Benefits
- Services can be acquired quickly, meaning that the
evolving requirements of the business can be
addressed almost immediately, giving an
organization a potential agility advantage.

- A properly implemented elastic system will

provision/de-provision according to application
demands, so if a particular business hasWarusawithana
Lakmal activity
spikes then the provision can be enabled to match
the demand and the capacity can be re-allocated.
Introduction to Elastic Cloud
Storage
Elastic Cloud Storage - Benefits
● Research and Development (R&D) Projects
- R&D activities are no longer hindered by a
requirement to secure a capex budget prior to a
project starting. Capability can simply be
provisioned from the cloud and released at the end
of the exercise.

Lakmal Warusawithana

Introduction to Elastic Cloud

Storage
 Elastic Cloud Storage - Benefits
● Testing and Deployment
- With most large-scale projects a size test needs to
be performed prior to final rollout. By taking
advantage of the elasticity of the cloud and creating
a full-scale avatar of the proposed production
system, realistic data and traffic volumes can be
provisioned and released as needed.

Lakmal Warusawithana

Introduction to Elastic Cloud

Storage
Storage Types

Lakmal Warusawithana

Source : https://thecustomizewindows.com/wp-content/uploads/2017/09/Object-Storage-vs-Block-Storage.png

Introduction to Elastic Cloud

Storage
Storage Types

File Storage vs Block Storage

https://www.youtube.com/watch?v=5EqAXnNm0FE

Object Storage vs File Storage vs Block Storage

https://www.youtube.com/watch?v=3r9RGJ0_Bls

Object Storage vs Block Storage

https://cloudacademy.com/blog/object-storage-block-storage/

Lakmal Warusawithana

Introduction to Elastic Cloud

Storage
Storage Needs vs Solutions
● Depending on our application, we have different
storage need

Databases
Frequent
Update SQL Based
DMNS
No-SQL DB Object Store
Spreadsheets

Google Drive
Amazon S3
Google Docs
Amazon Glacier
Google Blobstore
Google Cloud
Storage
XML
Never
Update
Static Files
Lakmal Warusawithana

Structured Unstructured/File Oriented

Introduction to Elastic Cloud

Storage
Google’s Storage Solution
● Google Cloud Storage is a RESTful online file
storage web service for storing and accessing your
data on Google's infrastructure.

● Google is making a terabyte of cloud storage available

for just $10

● which now offers the first 15 GB

per month for free. For $100
a month, Google offers as much Lakmal Warusawithana
space you could ever need:
10 terabytes or more
Introduction to Elastic Cloud
Storage
Introduction to Amazon Simple
Storage Service (S3)

Lakmal Warusawithana
SImple Storage Service

Lakmal Warusawithana

Introduction to Amazon Simple Storage Service (S3)

SImple Storage Service

Lakmal Warusawithana

Introduction to Amazon Simple Storage Service (S3)

S3 - Buckets and Objects

Lakmal Warusawithana

Introduction to Amazon Simple Storage Service (S3)

S3 - Statics Content Website

Lakmal Warusawithana

Introduction to Amazon Simple Storage Service (S3)

S3 - When to use?

Lakmal Warusawithana

Introduction to Amazon Simple Storage Service (S3)

 S3 - Use Case

Source https://aws.amazon.com/solutions/case-studies/yelp/

Introduction to Amazon Simple Storage Service (S3)

● Amazon S3 is safe, secure, highly scalable object
storage in the cloud.
● Amazon S3 is a very affordable solution for hosting
data on the cloud.
● You can store S3 securely store your information and
backup your critical data off-site
● Amazon S3 provides standards-based REST and
soap web services APIs, so you can programmatically
store, retrieve, and manage your data
Lakmal Warusawithana

Introduction to Amazon Simple Storage Service (S3)

● Most developers building applications on S3 used the
SDK, which wraps the underlying REST API.
● You can access S3 using command line interface or
AWS management console, which is a simple web
interface.
● Amazon S3 stores data as “objects” and objects are
stored within folders that are called “buckets”
● To store an object in S3, you upload a file you want to
store to a bucket.
● When you upload a file, you can set permissions on
Lakmal
the object so it’s private until you want Warusawithana
to share it.
● Buckets are the containers for objects

Introduction to Amazon Simple Storage Service (S3)

● You can have one or more buckets
● For each bucket, you can control access to bucket;
such as who can create, delete and list objects in that
bucket.

Lakmal Warusawithana

Introduction to Amazon Simple Storage Service (S3)

 Lakmal Warusawithana

Introduction to Amazon Simple Storage Service (S3)

 Lakmal Warusawithana

Introduction to Amazon Simple Storage Service (S3)

 Lakmal Warusawithana

Introduction to Amazon Simple Storage Service (S3)

 Lakmal Warusawithana

Introduction to Amazon Simple Storage Service (S3)

● After you create a bucket, you can not change its
name. That name is visible in the URL that point to
the objects stored in the bucket.
● You are not charged for creating a bucket, you are
only charged for storing objects in the bucket.
● You can choose a Region to
- Optimized Latency
- Minimize cost
- Address regulatory requirements
Lakmal Warusawithana

Introduction to Amazon Simple Storage Service (S3)

Upload Data to S3

Lakmal Warusawithana

Introduction to Amazon Simple Storage Service (S3)

 Lakmal Warusawithana

Introduction to Amazon Simple Storage Service (S3)

 Lakmal Warusawithana

Introduction to Amazon Simple Storage Service (S3)

 Lakmal Warusawithana

Introduction to Amazon Simple Storage Service (S3)

 Lakmal Warusawithana

Introduction to Amazon Simple Storage Service (S3)

● An object can be any kind of file, a text, photo, video,
or an application
● You can also drag and drop

Lakmal Warusawithana

Introduction to Amazon Simple Storage Service (S3)

View the object in S3 bucket

Lakmal Warusawithana

Introduction to Amazon Simple Storage Service (S3)

 Lakmal Warusawithana

Introduction to Amazon Simple Storage Service (S3)

Delete an object in S3 bucket

Lakmal Warusawithana

Introduction to Amazon Simple Storage Service (S3)

 Lakmal Warusawithana

Introduction to Amazon Simple Storage Service (S3)

Delete a S3 bucket

Lakmal Warusawithana

Introduction to Amazon Simple Storage Service (S3)

 Lakmal Warusawithana

Introduction to Amazon Simple Storage Service (S3)

 Lab Session
Introduction to Amazon Simple
Storage Service (S3)

Lakmal Warusawithana
Other Cloud-based Storage
Solutions
● General purpose file storage
○ Drop-box
○ AWS Elastic File Storage
● Backup purpose
○ AWS Glacier
● Rapid access
○ AWS DynamoDB
Lakmal Warusawithana

Introduction to Elastic Cloud

Storage
AWS Glacier

Lakmal Warusawithana

Introduction to Elastic Cloud

Storage
S3 Vs Glacier

Lakmal Warusawithana

Introduction to Elastic Cloud

Storage
S3 Lifecycle Policies

Lakmal Warusawithana

Introduction to Elastic Cloud

Storage
Use case

Lakmal Warusawithana

Introduction to Elastic Cloud

Storage
EC2 Instance Storage

Lakmal Warusawithana

Introduction to Elastic Cloud

Storage
Introduction to Amazon Elastic
Block Store (EBS)

Lakmal Warusawithana
Elastic Block Storage

Lakmal Warusawithana

Introduction to Elastic Cloud

Storage
Elastic Block Storage

Lakmal Warusawithana

Introduction to Elastic Cloud

Storage
Elastic Block Storage

Lakmal Warusawithana

Introduction to Elastic Cloud

Storage
Elastic Block Storage

Lakmal Warusawithana

Introduction to Elastic Cloud

Storage
EBS Performance

Lakmal Warusawithana

Introduction to Elastic Cloud

Storage
● Amazon EBS is like a virtual hard drive you can use
with your Amazon EC2 instances
● Its allows you to keep your data separate from your
computing instance.
● Its persists independently and you can attach it to
your instances when needed
● After a volume attached to a computing instances,
you can use it like any other physical hard drive and
place any kind of file system on it.
Lakmal Warusawithana

Introduction to Amazon Simple Storage Service (S3)

● When you use EBS to store your data, if computing
instance fails, you don’t have to worry about losing
your data because it isn’t housed on the instance - it’s
housed on your block storage volume.
● EBS volumes can be between 1 GB and 1 TB in size.
● They are created in a particular Availability Zone, or
AZ, which is a distinct physical location that contains
computing resources, similar to the data center.
● When you created EBS volume in an AZ, it will
automatically replicated within that Lakmal
zone toWarusawithana
prevent
data loss due to failure of any single hardware
component.
Introduction to Amazon Simple Storage Service (S3)
● EBS volume can only be attached to one instance at
a time, but many volume can be attached to a single
instance.
● When you attached, it will appear as a mounted
device and instance can interact with it similar to ant
hard drive or other storage device.
● You can attach multiple volumes and stripe your data
across them for increased I/O and throughput
performance.
● This is helpful for database-intensive applications
Lakmal Warusawithana
thats frequency encounter many random reads and
writes across the dataset.

Introduction to Amazon Simple Storage Service (S3)

AWS Storage Platform

Lakmal Warusawithana

Introduction to Elastic Cloud

Storage
Create EBS volume

Lakmal Warusawithana

Introduction to Amazon Simple Storage Service (S3)

 Lakmal Warusawithana

Introduction to Amazon Simple Storage Service (S3)

 Lakmal Warusawithana

Introduction to Amazon Simple Storage Service (S3)

 Lakmal Warusawithana

Introduction to Amazon Simple Storage Service (S3)

 Lakmal Warusawithana

Introduction to Amazon Simple Storage Service (S3)

 Lakmal Warusawithana

Introduction to Amazon Simple Storage Service (S3)

Attach EBS volume to Instance

Lakmal Warusawithana

Introduction to Amazon Simple Storage Service (S3)

 Lakmal Warusawithana

Introduction to Amazon Simple Storage Service (S3)

 Lakmal Warusawithana

Introduction to Amazon Simple Storage Service (S3)

 Lakmal Warusawithana

Introduction to Amazon Simple Storage Service (S3)

Creating Snapshot of EBS Volume

Lakmal Warusawithana

Introduction to Amazon Simple Storage Service (S3)

 Lakmal Warusawithana

Introduction to Amazon Simple Storage Service (S3)

 Lakmal Warusawithana

Introduction to Amazon Simple Storage Service (S3)

 Lakmal Warusawithana

Introduction to Amazon Simple Storage Service (S3)

 Lakmal Warusawithana

Introduction to Amazon Simple Storage Service (S3)

 Lakmal Warusawithana

Introduction to Amazon Simple Storage Service (S3)

 Lakmal Warusawithana

Introduction to Amazon Simple Storage Service (S3)

Increasing volume size and IOPS
● IOPS (Input/Output Operations Per Second,
pronounced eye-ops)
● which means the amount of read or write operations
that could be done in one seconds time. A certain
amount of IO operations will also give a certain
throughput of Megabytes each second, so these two
are related.
● Amazon EBS measures each I/O operation per
second (that is 256 KB or smaller) as one IOPS. I/O
operations that are larger than 256Lakmal
KB areWarusawithana
counted
in 256 KB capacity units.

Introduction to Amazon Simple Storage Service (S3)

 Lakmal Warusawithana

Introduction to Amazon Simple Storage Service (S3)

 Lakmal Warusawithana

Introduction to Amazon Simple Storage Service (S3)

 Lakmal Warusawithana

Introduction to Amazon Simple Storage Service (S3)

 Lakmal Warusawithana

Introduction to Amazon Simple Storage Service (S3)

 Lakmal Warusawithana

Introduction to Amazon Simple Storage Service (S3)

 Lakmal Warusawithana

Introduction to Amazon Simple Storage Service (S3)

 Lab Session
Introduction to Amazon Elastic
Block Storage (EBS)

Lakmal Warusawithana
Q&A

Lakmal Warusawithana
References
● https://aws.amazon.com/ebs/

● https://aws.amazon.com/s3

● https://youtu.be/_I14_sXHO8U

● https://youtu.be/77qLAl-lRpo

Lakmal Warusawithana