00 00 AWS - Fundamentals Instructor Guide

Download as pdf or txt
Download as pdf or txt
You are on page 1of 144
At a glance
Powered by AI
The key takeaways from the document are an understanding of what cloud computing is, the advantages of cloud computing, cloud segments, deployment models, key cloud companies and services, cloud security, and cloud use cases.

Some of the key advantages of cloud computing mentioned are reduced costs, increased scalability and flexibility, easy access to services, and no upfront infrastructure costs.

The different cloud segments mentioned are Infrastructure as a Service (IaaS), Platform as a Service (PaaS), and Software as a Service (SaaS).

Fundamentals

of Cloud Computing
Key Takeaways
• Fundamental understanding of what Cloud Computing is
• Understanding of the key advantages of Cloud
• Understanding of Cloud Segments
• Understanding of Cloud Deployment Models
• Knowledge of some key Cloud companies
• Knowledge of some Cloud services
• Cloud Security
• Cloud use cases

www.cloudthat.in
Roadmap
• What is Cloud Computing, why it matters
• Traditional IT Infrastructure
• Cloud Infrastructure and Cloud Advantage
• Examples of Cloud Advantage
• Cloud Companies
• Examples of Cloud Services
• Use Cases
• Cloud Segments: IaaS, PaaS, SaaS
• Cloud Deployment Models
• Cloud Security
www.cloudthat.in
Growth Predictions
• Gartner: Worldwide Public Cloud Revenue to Grow 21.4 Percent in 2018
• Gartner: Infrastructure as a Service (IaaS) forecast to grow 35.9 percent
in 2018 to reach $40.8 billion
• IDC: By 2020, public cloud services spending is expected to reach more
than $162 billion
• IDC: By 2018 Cloud-based spending to be 60% of all IT infrastructure,
and 60–70% of all Software
• Wikibon: Enterprise cloud spending is growing at a 16% (CAGR)
between 2016 and 2026
• BDO Survey: 74% of Tech CFOs say cloud computing will have the most
measurable impact on their business

www.cloudthat.in
Job Predictions
• Forbes: Cloud Computing to create 18 million Jobs
• US Market - Amazon, EMC and VMWare aggressive hirers on cloud
computing jobs in the last 12 months
• Certifications in cloud computing pay an average salary of $112,955 per
year
• Global Market - The global public cloud market is expected to grow at a
CAGR of 22 %, its size expected to be 178 billion in 2018
• More than 3,800 cloud-related jobs are listed on a daily basis

www.cloudthat.in
What is Cloud Computing

“Cloud Computing is the delivery of computing as a service


rather than a product”

www.cloudthat.in
1975 312 T Model
495 bhp
Red Bull RB9
750 bhp
In 40 years we improved roughly 50% in Horse Power
Altari 8800, the first commercially available computer launched in 1975. 2
MHz CPU, 256 bytes memory, blinkenlights & toggle switches for input-
output
iPhone X

Quad-core 2.39 GHz CPU, 256 GB of storage. That’s 140,000% faster and over 2
billion times more storage. Consumes a fraction of the electricity.
Evolution of Software

Mainframe PC Post-PC

Centralize Software as a Software as a


Product Service
d • Different vendors • Age of internet
provides • Single client, multiple
Softwar hardware/software

devices
Vendor provides a
e • Software
management is

hosted solution
Hardware and
• Same vendor distributed
software for running
provides • Client is responsible the application
hardware/softwar for managing provided by the
e software vendor
www.cloudthat.in
• Software
management is
Roadmap
• What is Cloud Computing, why it matters
• Traditional IT Infrastructure
• Cloud Infrastructure and Cloud Advantage
• Examples of Cloud Advantage
• Cloud Companies
• Examples of Cloud Services
• Use Cases
• Cloud Segments: IaaS, PaaS, SaaS
• Cloud Deployment Models
• Cloud Security
www.cloudthat.in
Traditional IT Infrastructure

Cellphones Computers Tablets

Your Applications Application Layer

Application Low Level Software


Database WebServer
Server Resources

Computation Storage Bandwidth


Low Level Hardware
Resources

Networking
Datacenter Machines
Equipment

www.cloudthat.in
IT Infrastructure - Issues

Tightly coupled to
geographic
location
Application Layer

Upfront licensing
Needs good Low Level Software
software
cost
maintenance team Resources

Very hard to
estimate need in
Becomes cheaper
every month
Low Level Hardware
advance
Resources

Needs a lot of Has physical Needs good


capital upfront limitations network engineers

www.cloudthat.in
IT Infrastructure - Issues

Application Layer

Low Level Software


Resources

Low Level Hardware


Resources

www.cloudthat.in
IT Infrastructure – Pyramid of Effort

Application Layer

Low Level Software


Resources

Low Level Hardware


Resources

www.cloudthat.in
Use Case Study
• Microsoft Office to Office 365

www.cloudthat.in
Hardware Requirement

• Infrastructure
– Application and storage on Company Data Centers
– Thousands of machines required
– Huge upfront costs for this infrastructure
– Huge operations/deployment team
– Huge hardware installation and maintenance team
– Lot of Network engineers
– Network Operations Center (NOC): 24/7 monitoring of the service

www.cloudthat.in
Software Development

• Software engineers that can write distributed


applications
• Applications need to be fault scalable and fault
tolerant
• Needs to know network topology and code for it
• Overall developers job becomes tougher

www.cloudthat.in
Software Testing

• From single machine testing to testing on large


cluster
• Test clusters usually smaller than production clusters
• Difficult as problems might manifest only at large
scale
• Testing for different clients

www.cloudthat.in
Marketing/Business Model

• One time fixed cost vs Subscription model


• Recurring cost to maintain datacenters

www.cloudthat.in
Project Management

• From many years cycle to rapid iterations


• Agile vs Waterfall
• Ability to try new features faster

www.cloudthat.in
• Many other things change…
• Overall its not easy to develop and maintain hosted
applications

www.cloudthat.in
Roadmap
• What is Cloud Computing, why it matters
• Traditional IT Infrastructure
• Cloud Infrastructure and Cloud Advantage
• Examples of Cloud Advantage
• Cloud Companies
• Examples of Cloud Services
• Use Cases
• Cloud Segments: IaaS, PaaS, SaaS
• Cloud Deployment Models
• Cloud Security
www.cloudthat.in
Cloud Infrastructure

Needs a lot Needs


Has
of capital good
physical
upfront network
limitations
engineers

Amazon Elastic
Google
Azure VM Compute Cloud Networking
Datacenter Machines Compute Engine
(EC2) Equipment

www.cloudthat.in
Cloud Infrastructure

Very hard Becomes


to estimate cheaper
need in every
advance month

Amazon Simple
Azure Blob
Amazon Elastic Low Level Hardware
Storage Service Load Balancer
(S3)
Storage
(ELB) Resources

Amazon Elastic
Google
Azure VM Compute Cloud
Compute Engine
(EC2)

www.cloudthat.in
Cloud Infrastructure

Needs good
Upfront
software
licensing maintenance
cost team

Amazon
Azure SQL
Relational Google App Low Level Software
Database Engine
Service Resources

Amazon Simple
Azure Blob
Amazon Elastic Low Level Hardware
Storage Service Load Balancer
(S3)
Storage
(ELB) Resources

Amazon Elastic
Google
Azure VM Compute Cloud
Compute Engine
(EC2)

www.cloudthat.in
Cloud Infrastructure
Tightly
coupled to
geographic
location

Google Apps
Microsoft Million other
(Mail, Calender,
Exchange Online apps Application Layer
etc.)

Amazon
Azure SQL
Relational Google App Low Level Software
Database Engine
Service Resources

Amazon Simple Amazon Elastic


Storage Service
Azure Blob
Load Balancer Low Level Hardware
Storage
(S3) (ELB) Resources
Amazon Elastic
Google
Azure VM Compute Cloud
Compute Engine
(EC2)

www.cloudthat.in
IT Infrastructure – Cloud Advantage

Application Layer

Low Level Software


Resources

Low Level Hardware


Resources

www.cloudthat.in
Key Attributes of Cloud Computing

• Self-service
• Pay-per-use/Utility model
• Elasticity
• High Scalability
• Fully virtualized/automated
• Uses Internet Protocols and Technologies (HTTP,
REST, SOAP, etc.)

www.cloudthat.in
Roadmap
• What is Cloud Computing, why it matters
• Traditional IT Infrastructure
• Cloud Infrastructure and Cloud Advantage
• Examples of Cloud Advantage
• Cloud Companies
• Examples of Cloud Services
• Use Cases
• Cloud Segments: IaaS, PaaS, SaaS
• Cloud Deployment Models
• Cloud Security
www.cloudthat.in
Examples of Cloud Advantage
• RedBus.in • Zynga.com
– Started in 2006 – Started in 2007 now a $6.6 billion
– Today has 1.2 million page company
views every day. – Zynga has > 250m monthly active
– Now uses Cloud to scale. users globally
– Owns 7 of top 10 games on
Facebook.com

www.cloudthat.in
Zynga Story

www.cloudthat.in
Failed to Scale

• Friendster:
– Had three million users within the first few months
– “We didn’t solve the first basic problem: our site didn’t work”, Kent
Lindstrom, Ex-CEO, Friendster
– “All they had to do was keep the damned servers up and running,” Matt
Cohler, Ex-Facebook executive

• Cuil (pronounced ‘cool’)


– Was launched as “Google Killer” search engine
– Greater number of pages indexed than Google
– The hype created way too much traffic at launch
– Caused what company said to be a “serious file corruption” due to
overloading
www.cloudthat.in
Adapted to Scale: Animoto

Soundtrack

Video

www.cloudthat.in
Adapted to Scale: Animoto

CPU

Good
bandwidth

www.cloudthat.in
Adapted to Scale: Animoto

CPU

Good
bandwidth

www.cloudthat.in

40
Cloud Advantage: Redundancy (AWS Regions)

EU
US-
West-
1
US-
Japan
US- East-1
West- China
2

Singapor
e

SA
Australi
a

www.cloudthat.in
AWS Data Centers

A B
A B C
C
A B
A B
A B
C A B
C
C
C

A B

A B

C A B

www.cloudthat.in
Revisit: Hosting Applications using Cloud

• Cloud Infrastructure saves a lot of time, money and


energy
• Less manpower needed for maintaining the IT
infrastructure
• Developers are aided by many cloud services
• Testers can have production size clusters to test
• Business models are simpler/Startups need less
capital to go live

www.cloudthat.in
Things to Watch Out For

• Security and Privacy


• You need a good internet connection to use public
cloud
• Billing can sometimes be complex
• Costing is totally in the hands of the provider
• It changes a lot of inherent roles and responsibilities
in an organization
• Lack of standards
www.cloudthat.in
Roadmap
• What is Cloud Computing, why it matters
• Traditional IT Infrastructure
• Cloud Infrastructure and Cloud Advantage
• Examples of Cloud Advantage
• Cloud Segments: IaaS, PaaS, SaaS
• Cloud Companies
• Examples of Cloud Services
• Use Cases
• Cloud Deployment Models
• Cloud Security
www.cloudthat.in
Cloud Segments

Microsoft
Million other apps
Exchange Online

Software as Google Apps


SalesForce (Mail, Calender,
a Service etc.)
(SaaS)
Azure SQL Azure App Fabric

Google App
Amazon Elastic
Platform as a Service (PaaS) Engine
Beanstalk

Amazon Simple Amazon Elastic


Storage Service Compute Cloud
(S3)
Infrastructure as a Service (IaaS) (EC2)

Google Compute Rackspace Cloud


Engine Servers

www.cloudthat.in
IaaS

Data Center

Load Balancer

Execution Machines

Machine 1 Machine 2

Webserver Webserver

Backend Backend

Cloud Database Cloud Storage

Database

www.cloudthat.in
PaaS

Geographic Location of Service

Load Balancer

Execution Machines

Cloud Database Cloud Storage

www.cloudthat.in
Key Benefits - IaaS

• Very high flexibility


• Drastic reduction in upfront capital investment
• Scale easily and rapidly
• Pay for only the resources used
• Drastic reduction in IT Support, networking and
system admin staff required for maintenance
• Takes care of NON-core but critical infrastructure
components
www.cloudthat.in
Key Benefits - PaaS

• Needs even less upfront capital


• Requires less knowledge and skill to develop and deploy
applications
• Less flexibility
• Pay for only the resources used
• Easy and rapid scaling
• Rapid time to market
• Takes care of NON-core but critical platform components
• Drastic reduction in IT staff required for software
maintenance www.cloudthat.in
Key Benefits - SaaS

• No installation and maintenance requirements


• Pay for only the resources used
• No upfront licensing costs
• Ability to access from anywhere
• Ability to access from multiple devices
• Usually better SLAs then one can achieve on their
own

www.cloudthat.in
Cloud Deployment Models

• Public
• Private
• Community
• Hybrid

www.cloudthat.in
Cloud Deployment -
Public Internet

• Public cloud -- The cloud


infrastructure is made
available to the general Public Cloud
public or a large industry
group and is owned by an
organization selling cloud
services

Organization 2 Organization 1
Cloud Deployment -
Private

Organizational
Boundary
• Private cloud -- The cloud
infrastructure is operated
solely for an organization.
It may be managed by
the organization or a Private Cloud
third party and may exist
on premise or off
premise.
Cloud Deployment -
Private

Organizational Boundary
Private Cloud Layer

Virtual Virtual Virtual Virtual Virtual Virtual


Instance Instance Instance Instance Instance Instance

Machine Virtualization Layer (Hypervisor) Machine Virtualization Layer (Hypervisor)

Physical Server Physical Server


Cloud Deployment -
Community

Organization - 1
Boundary
• Community cloud -- The
cloud infrastructure is
shared by several
organizations and
Community
supports a specific Cloud
community that has
shared goals

Organization-2
Boundary
Cloud Deployment -
Organizational
Hybrid
Boundary

Private
Cloud

• Hybrid cloud -- The cloud


infrastructure is a App
composition of two or more
clouds (private, community,
or public) that remain
unique entities but are
bound together by
standardized or proprietary Internet
technology that enables
data and application
portability
Public
Cloud
Roadmap
• What is Cloud Computing, why it matters
• Traditional IT Infrastructure
• Cloud Infrastructure and Cloud Advantage
• Examples of Cloud Advantage
• Cloud Companies
• Examples of Cloud Services
• Use Cases
• Cloud Segments: IaaS, PaaS, SaaS
• Cloud Deployment Models
• Cloud Security
www.cloudthat.in
Public Cloud Companies
SaaS PaaS IaaS
SalesForce (2001) Google App Engine (2008) Amazon Web Services
(2006)
Microsoft Office 365 (2011) IBM SmartCloud Foundation Azure Infrastructure
(2011) Services (2012)
Microsoft Exchange Online Microsoft Azure (2008) IBM SmartCloud Enterprise
(2008) (2011)
Google Apps (2007) Amazon Elastic Beanstalk Rackspace Cloud Servers
(2010) (2009)
Force.com by Google Compute Engine
SalesForce.com (2006) (2012)
HP Cloud Services (2012)
Linode (2002)
Joyent Cloud (2004)
www.cloudthat.in
Private Cloud Companies

IBM SmartCloud
Provides Hardware
HP Cloud EMC
and Software

OpenStack (backed by Rackspace, Apache Cloud Stack (backed by Citrix,


NASA, HP, etc.) etc.)
Provides Only
Eucalyptus (backed by AWS) VMware
Software

www.cloudthat.in
IaaS Gartner Magic Quadrant - 2017

www.cloudthat.in
Roadmap
• What is Cloud Computing, why it matters
• Traditional IT Infrastructure
• Cloud Infrastructure and Cloud Advantage
• Examples of Cloud Advantage
• Cloud Companies
• Examples of Cloud Services
• Use Cases
• Cloud Segments: IaaS, PaaS, SaaS
• Cloud Deployment Models
• Cloud Security
www.cloudthat.in
Cloud Storage Services

Data Center 2
Mypic.jpg

Report.doc

Data Center 1
Store Mypic.jpg
Mypic.jpg
Store Report.doc
Report.doc

My App List
Data Center 3
Mypic.jpg
Get Mypic.jpg
List Report.doc
Result
……….
Delete Mypic.jpg
……….

www.cloudthat.in
AWS S3: Availability and Durability

• 99.99% availability (52 mins of downtime a year)


• 99.999999999% durability (with 10,000 objects a
loss of a single object in 10,000,000 years)

www.cloudthat.in
Cloud Compute Services

Give me 10 machines

My App
I only need 5 machines

Done

www.cloudthat.in
AWS Elastic Cloud Compute Features

• Start publicly accessible servers in mins using web


services
• Variety of machine types:
– From 1-64 cores
– 613 MB - 500 GB RAM
– Linux (RedHat, openSuSE, Fedora, Debian, Cent OS, Gentoo ), OpenSolaris,
Oracle Linux, and Windows Server

• A lot of preconfigured machine types


• Pay per hour for the instance and external bandwidth

www.cloudthat.in
Microsoft SQL Azure
• Extends SQL Server capabilities to the cloud
• Provision in minutes
• Self-managing, automatic failover
• High Availability
• Auto Scaling; service scales as your data grows
• Pay as you go
• Easy replication across geographic locations
• Geographic redundancy
• Spend time on your database design; Not on physical
implementation
www.cloudthat.in
Azure SQL

www.cloudthat.in
Google Application Engine (GAE)

• To develop and deploy apps in Java, Python, and Go


• Auto scales up or down according to usage
• Only pay for the resources you use

www.cloudthat.in
Roadmap
• What is Cloud Computing, why it matters
• Traditional IT Infrastructure
• Cloud Infrastructure and Cloud Advantage
• Examples of Cloud Advantage
• Cloud Companies
• Examples of Cloud Services
• Use Cases
• Cloud Segments: IaaS, PaaS, SaaS
• Cloud Deployment Models
• Cloud Security
www.cloudthat.in
Use Case: My Exchange is Down Again
• Solution: Move to Google Apps or Microsoft Exchange
Online
• Reduced deployment and maintenance cost
• Simplified high availability and disaster recovery (99.99%
availability, auto backup and recovery)
• Greater mobility and flexible access
• Mailbox duplicated in multiple datacenters – No need for
backups
• Unlimited mailbox size (or 25 GB) and archiving
• Auto spam and virus protection
• Cons: Migration of existing user accounts requires some
cloud knowledge
www.cloudthat.in
Price Comparison
• Microsoft
• In-house Google Apps Exchange
Online
Exchange
Server One • Rs 2500 per
Time/Per 3 Years • Rs 3000
Month Aggregate user per year per user
Two Servers 150,000 150,000 • 100 users = per year
Backup Device 10,000 10,000 Rs 7,50,000 • 100 uses =
Good 16 Rs
Bandwidth 5,000 1,80,000 14 9,00,000
Power 2,000 72,000 12
IT person 20,000 7,20,000 10
UPS 20,000 20,000 8
6
100 users
license 4
2,00,000 2,00,000
2
Total 13,52,000 0
Inhouse Google Apps Microsoft
Exchange Server Exchange Online
www.cloudthat.in
Cloud Migration

www.cloudthat.in
Cloud Migration
Includes:
• Financial Assessment (TCO/ROI
Analysis)
• How much will it cost to run?
• Security and Compliance
Assessment
• Technical Assessment
– Choosing the right
candidate
– Migrating Licensed
products
– Identifying tools you can
reuse
– Functional/Architectural
assessment
www.cloudthat.in
Cloud Migration
Includes:
• Get your feet wet with Amazon
Web Services
– Learning AWS
– Build reference architecture
– Be aware of the security
features
• Build a Prototype/Pilot
– Build support in your
organization
– Validate the technology
– Test legacy software in the
cloud
– Perform benchmarks
www.cloudthat.in
Cloud Migration
Includes:
• Learning about different
database storage options
available today
• Uploading/Moving your data in
Batches
• Analyzing your
database/datasets
• Build necessary tools and scripts
to migrate data
• Security of your data
(Encryption)

www.cloudthat.in
Cloud Migration

Includes:
• Decide a strategy
(Forklift/Hybrid)
• Automate your processes
by creating AMIs
• Build “cloud-aware” layers
of code for tightly coupled
apps

www.cloudthat.in
Cloud Migration

Includes:
• Leveraging other AWS
services
• Implementing “Elasticity”
• Automating in-cloud
software development and
deployment lifecycle
• Increasing high availability
by leveraging multiple
Availability Zones
• Hardening Security

www.cloudthat.in
Cloud Migration
Includes:
• Understanding your usage patterns
and managing cloud resources more
effectively
• Optimizing to increase Performance,
improve efficiency and save further
costs
• Advanced Monitoring and telemetry
• Ensuring enhanced security
• Refactoring applications as
necessary
• Leverage AWS Best Practices

www.cloudthat.in
Roadmap
• What is Cloud Computing, why it matters
• Traditional IT Infrastructure
• Cloud Infrastructure and Cloud Advantage
• Examples of Cloud Advantage
• Cloud Companies
• Examples of Cloud Services
• Use Cases
• Cloud Segments: IaaS, PaaS, SaaS
• Cloud Deployment Models
• Cloud Security
www.cloudthat.in
Deployment Models

• Hybrid
• Inter Geographic
• Intra Geographic

www.cloudthat.in
Roadmap
• What is Cloud Computing, why it matters
• Traditional IT Infrastructure
• Cloud Infrastructure and Cloud Advantage
• Examples of Cloud Advantage
• Cloud Companies
• Examples of Cloud Services
• Use Cases
• Cloud Segments: IaaS, PaaS, SaaS
• Cloud Deployment Models
• Cloud Security
www.cloudthat.in
Cloud Security

• Security at various layers


– Infrastructure
– Platform
– Software
– Application

• Shared Security Model

www.cloudthat.in
Questions?

www.cloudthat.in
AWS Certification Track

SysOps
Solutions Architect Developer
Administrator

AWS Certified Solutions


AWS Certified DevOps Engineer - Professional
Professional Level Architect - Professional

AWS Certified SysOps


AWS Certified Solutions AWS Certified
Administrator -
Associate Level Architect - Associate Developer - Associate
Associate

www.cloudthat.in
Azure Certification Track

Developing Microsoft Azure Solutions 70 - 532

Implementing Microsoft Azure Infrastructure Solutions 70 - 533

Architecting Microsoft Azure Solutions 70 - 534

www.cloudthat.in
Amazon S3
Amazon Simple Storage Service

• Highly scalable, limitless storage space


• Storage as a Service: through Web Services API
• Very high data durability: no need for backup in most
cases
• Fast and highly available
• Pay as you go model with very competitive rates

www.cloudthat.in
Amazon S3 Basic Operations

Data Center 2
Mypic.jpg

Report.doc
Store Mypic.jpg

Data Center 1

Store Report.doc Mypic.jpg

Report.doc

My App List
Data Center 3
Mypic.jpg

List
Get Mypic.jpg Report.doc
Mypic.jpg
Report.doc

Delete Mypic.jpg

www.cloudthat.in
Consistency Model for Distributed Storage

Eventual Consistency
Data Center 2
Mypic.jpg
Store Mypic.jpg
Data Center 1
Mypic.jpg
Response (OK)
My App
Data Center 3

Get Mypic.jpg

• Write and immediate read might return 404


NOT_FOUND
• Overwrite and immediate read might return old
key www.cloudthat.in
Consistency Model for Distributed Storage

Eventual Consistency
Data Center 2
Mypic.jpg
Delete Mypic.jpg
Data Center 1
Mypic.jpg
Response (OK)
My App
Data Center 3

Get Mypic.jpg Mypic.jpg

• Delete and immediate read might return


key www.cloudthat.in

• Delete then immediate list might return


S3 Buckets

• Logical containers of objects


• Has some container properties
• Bucket names are global and needs to be unique
• 100 Buckets per AWS account
• Bucket names has plenty of limitations
• From 3 to 63 characters
• all lower case
• Cannot start with a “.”
• Must be DNS compliant
• Etc. (http://docs.amazonwebservices.com/AmazonS3/latest/dev/BucketRestrictions.html)

www.cloudthat.in
Amazon S3: Namespace

cloudthat.in/courses/level-1
Global part Custom Part

bucketname.s3.amazonaws.com/objectName
Global Part Custom Part

OR
s3-ap-southeast-1.amazonaws.com/bucketname/objectName

Global Part Custom Part

www.cloudthat.in
S3 Keys and Objects

• Each bucket can contain unbounded number of


objects
• Object contents are opaque to S3 and can be
anything
• Normal objects upto 5GB, multipart upload objects
upto 5TB.
• Each object is represented by unique key
• Keys can be upto 1024 bytes
• Keys are not hierarchical
www.cloudthat.in

• Can be simulated to be hierarchical by adding “/” to


Bucket ACL
• List Objects
Bucket ACL
• Write Objects
• Read Bucket Permissions
• Edit Bucket Permissions
Object Object Object
1 2 3

www.cloudthat.in
Object ACL
• Read Object
Object ACL
• Write Object
• Read Object Permissions
• Edit Object Permissions

www.cloudthat.in
ACL: GRANTS

• Owner
• Everyone (Public)
• Other AWS account/email ID

www.cloudthat.in
ACL Pros and Cons
Pros Cons
• Easy to implement • Very limiting in scope
• Secure by default • Not-hierarchical
• Mistakes take changing
ACL to all objects

www.cloudthat.in
Bucket Policies

• New way to grant and deny access to S3 resources

www.cloudthat.in
Bucket Policy
{ Allows full access to
"Version": "2008-10-17",
"Id": "S3PolicyId1",
"Statement": [ • Two given users ONLY
{
"Sid": "IPAllow", when
"Effect": "Allow",
"Principal": {
"AWS":[”8975-2909", ”2091-0929"] • IP-addresses in the given
},
"Action": "s3:*", range
"Resource": "arn:aws:s3:::bucket/*",
"Condition" : {
"IpAddress" : { • CIDR notation for ip-
}
"aws:SourceIp": “50.50.50.0/24"
address range described
}

}
] in RFC 2632

www.cloudthat.in
Pre Signed URLs

• Pre sign the URL


• Can specify expiry time

www.cloudthat.in
Reduced Redundancy Storage

• Less copies of your data


• Good for data that can be regenerated easily
• Less cost
• Notification when object’s all copies are lost

www.cloudthat.in
Infrequent Access Storage Option

• High durability, low latency, and high throughput of


S3 Standard
• $0.01 / gigabyte charge for retrieval (in addition to
the usual data transfer and request charges)

www.cloudthat.in
S3 Multipart upload

• Declare the object, upload in multiple parts


• Explicitly complete after all uploads
• Parts can be uploaded in parallel for high throughput
• Smaller parts makes retry on network failures less
costly
• Upload objects of unknown size (streaming)

www.cloudthat.in
S3 Data Encryption
• Uses AES-256
• PUT an object and
request encryption (in an
HTTP header)
• GET remains the same.
Response contains extra
header
• Bucket policies to allow,
mandate, or forbid
Source: http://aws.typepad.com/aws/2011/10/new-amazon-s3-server-side-encryption.html
encryption at the bucket www.cloudthat.in
S3 Versioning

• Instead of overwrite, set versions


• Object accessed by Key gives latest object
• Object of previous versions can be accessed by
“Key” + “Version”

• When you delete, object is NOT deleted, just delete


marker is put.

www.cloudthat.in
Object Lifecycle Management

• Manage the lifecycle of every object in a bucket


• Managing is via a set of rules with predefined actions

• Actions include:
• Transition actions – transition to another storage
(Infrequent access storage or Glacier)
• Expiration actions – expiration of objects

www.cloudthat.in
Static Website Hosting

• Web pages with static content


• Doesn’t support dynamic website & server-side
scripting
• Access S3 hosted website with custom domain name
• Mention the index & error page for the website
• Configure a webpage redirect
– Region-specific website endpoint
– REST endpoint

www.cloudthat.in
S3 Pricing
Asia Pacific (Singapore)
• Pay for storage in • $0.025 Per GB
GB/month
• $0.120 per GB (first GB free)
• Pay for bandwidth out
of AWS in GB • PUT, COPY, POST, or LIST
Requests $0.01 per 1000 requests
• GET and all other Requests -
• Pay for requests $0.01 per 10,000 requests

aws.amazon.com/pricing/s3/
www.cloudthat.in
Requester Pays Buckets

• Bucket owner pays for the storage of data


• Requester pays for the request & data downloaded
from the bucket
• Anonymous access to the bucket is prohibited
• Ideal for sharing publicly accessible data from S3
– Useful when providing large-datasets
– requesters must include “x-amz-request-payer” header in their requests

www.cloudthat.in
Cross-Region Replication

• Bucket-level feature for copying of objects across


buckets in different AWS regions
• The source and destination buckets must be
versioning-enabled
• Assign IAM permissions for S3 to replicate objects
• Ideal to minimize latency for content retrieval,
compliance requirements

www.cloudthat.in
Transfer Acceleration

• Fast, easy, and secure transfers of files to S3 bucket


• Utilizes AWS CloudFront’s global edge locations to
route data to S3
• Incur additional data transfer charges
• Transfer Acceleration enabled bucket names should
be DNS compliant
• Amazon S3 Transfer Acceleration Speed Comparison
tool

www.cloudthat.in
Questions?

www.cloudthat.in
Amazon EC2
EC2 Overview

Give me 10 machines

I only need 5 machines


My App

Done

www.cloudthat.in
Questions?

www.cloudthat.in
Virtual Servers

Virtual EC2 Virtual EC2 Virtual EC2


Instance Instance Instance

Machine Virtualization Layer (Hypervisor)

Physical Server

www.cloudthat.in
Instance Types

• Different combination of CPU, Memory and Hard-disk


are available
• CPU defined in terms of EC2 Compute Units
– One EC2 Compute Unit provides the equivalent CPU capacity of a 1.0-1.2
GHz 2007 Opteron or 2007 Xeon processor

www.cloudthat.in
EC2 Instance Types

Family Description
Standard General purpose CPU – Memory.
Micro Very low CPU, allows small bursts. Only good for websites and
applications that doesn’t need sizeable amount of CPU/memory.
High-CPU Higher CPU than RAM.
High- Higher Memory than CPU.
Memory
Cluster Very large amount of CPU coupled with increased networking
Compute performance. Ideal for High Performance Compute (HPC)
applications and other demanding network-bound applications.
Cluster GPU Graphics processing units (GPUs), with high CPU and increased
network performance. Ideal for applications that needs high
rendering power as well as HPC applications

www.cloudthat.in
EC2 Instance Types
Linux Windows
Name Memory Compute Units Storage Platform I/O Perf API Name
cost cost

Standard Small 1.7 GB 1 (1 core x 1 unit) 160 GB 32/64- Moderate m1.small $0.08 $0.115
bit per hour per hour

Standard 3.75 GB 2 (1 core x 2 410 GB 32/64- Moderate m1.medium $0.16 $0.23 per
Medium units) bit per hour hour

Standard Large 7.5 GB 4 (2 cores x 2 850 GB 64-bit High m1.large $0.32 $0.46
units) per hour per hour

Standard Extra 15 GB 8 (4 cores x 2 1690 64-bit High m1.xlarge $0.64 $0.92 per
Large units) GB per hour hour

Micro 0.6 GB 2 (only for short EBS 32/64- Low t1.micro $0.02 $0.03
bursts) only bit per hour per hour

High-Memory 17.1 GB 6.5 (2 cores x 420 GB 64-bit Moderate m2.xlarge $0.45 $0.57 per
Extra Large 3.25 units) per hour hour

High-Memory 34.2 GB 13 (4 cores x 3.25 850 GB 64-bit High m2.2xlarge $0.90 $1.14 per
Double Extra units) per hour hour
Large

http://www.ec2instances.info/
www.cloudthat.in
EC2: Where is my DATA?

Machine
Failure

My
EC2
EC2 EC2 EC2 EC2 My
EC2
EC2 EC2
Instance Instance Instance Instance Instance Instance

Machine Virtualization Layer Machine Virtualization Layer


(Hypervisor) (Hypervisor)

Physical Server Physical Server

www.cloudthat.in
EC2: Where is my DATA?

EBS EBS
Machine
Failure

My
EC2
EC2 EC2 EC2 EC2 EC2 EC2
Instance Instance Instance Instance Instance Instance

Machine Virtualization Layer Machine Virtualization Layer


(Hypervisor) (Hypervisor)

Physical Server Physical Server

www.cloudthat.in
Customer A
EBS Architecture
Single Availability Zone
EC2 Instance
EBS Storage EBS Storage
Volume A1 Machine n1 Machine m1

Volume A2 Volume A1 Volume A1

Volume A3 Volume B1 Volume B1

Customer B

EC2 Instance

Volume B1 EBS Storage EBS Storage


Machine n2 Machine m2
Volume B2

Volume B3
Volume A3 VolumeA 3

Note: This is a best guess and have no


insight into actual EBS architecture
www.cloudthat.in
Elastic Block Storage (EBS)

• Virtual hard disk that persists beyond and


independent of EC2 instance
• Behaves like raw block device
• Performance similar to local hard disk
• More durable/available than individual hard disks
• Can be from 1GB to 16TB in size
• Easy to share data between different instances

www.cloudthat.in
Provisioned IOPS

• Pre-provision IOPS when allocation volume in


addition to size
• Costs extra

www.cloudthat.in
Persistence of Data
10x durability of a single hard drive

EC2 Instance Elastic Block EC2


Storage (EBS) Instance
EBS EBS EBS
Volume Volume Volume
Local EBS
Drive Volume

No durability of
local drive

EBSEBS
EBS
Snapsho
Snapsho
Snapshot
t t

S3-like storage

99.999999999% durability guarantee


www.cloudthat.in
EBS is NOT S3

EBS Amazon S3
Paradigm Simulates File Object Store
System
Durability 10x hard-disk 99.999999999%
Access ONLY from EC2 Web-based access
Consistency Similar to hard-disk Eventually
consistency
Availability/Redunda Replicated inside a Replicated across
ncy data-center data-centers
Pricing Per Allocated Size of Per stored GB
Volume Bandwidth
IO operations Access operations

www.cloudthat.in
EBS Pricing

• Pay for allocated storage


– $0.12 per allocated GB per month.

• Pay for provisioned IOPS


– $0.072 per provisioned IOPS-month

• Snapshots
– $0.05 per GB-month of data stored

Note: Higher pricing for provisioned IOPS


volumes
Values are for Singapore region

www.cloudthat.in
Amazon Machine Image (AMI)
Suse Amazon RedHat Windows
Linux Linux Linux Server

EC2 Instance EC2 Instance

Custom
Custom
AMI
AMI
Root Drive Root Drive
Apache MySQL Apache MySQL
PHP PHP
Custom Custom
Application Application

www.cloudthat.in
Local drive Instance Store AMI (legacy style)

1) Instance bootup 4) Reboot

EC2 Instance

Instance Store

Amazon
Local Drive Linux AMI
Apache MySQL
PHP
Custom
Application 2) Image stored in Instance Store
loaded
3) Install custom software
5) On machine failure, all stored data is lost
www.cloudthat.in
EBS Backed AMI (newer style)

1) Instance bootup 4) Reboot

EC2 Instance

EBS Drive

EBS Drive EBS


Apache
PHP MySQL
Custom
Application 2) Image loaded

3) Install custom software


5) On machine failure, all data stored on EBS drive retains
www.cloudthat.in
Which is better?

Instance Store AMI EBS-Backed AMI

Default Legacy offering Added in 2009; default now

Root Drive Failure Behavior Data NOT persisted Data persists failures

Boot Time Few mins Usually less than a min


Cost IO is free IO costs as per EBS charges

Performance More stable IO. Traditional IO Usually better performance


bottlenecks than local disk.
IO variable as over network.

www.cloudthat.in
EC2 States

Launch

Pending

Reboot

Running

Stop Terminate
Stopped Terminated

Start
www.cloudthat.in
Security Groups

Internet

Security Group

EC2 Instance Port 80

www.cloudthat.in
Security Groups

Internet

Web server Security Group

Web Web
Server Server
Port 80

Web
Server

www.cloudthat.in
Security Groups
Database
Security Group

Database Internet
Port 3306:
Only
Webserver Web server Security Group
SG
Web Web
Server Server
Port 80:
Web Everyone
Server

www.cloudthat.in
Security Groups

• Inbound access control only (VPC with Outbound)


• Rules Allowed:
– Protocol {http, ssh, ftp, etc.}
– Port Range
– IP address Range

www.cloudthat.in
Key-Pair Security Credentials
1) Generate public-private
key pair.
EC2 Instance
2) Keep the private key
safely
3) At instance creation
time, EC2 installs the
public key into new Local Drive
instances
4) After instance is
created, login using
private key

Local Machine

www.cloudthat.in
EC2 Pricing

• Cost per instance-hour


– Remember partial instance hours are charged as one hour

• Bandwidth out of EC2 instance


– Data transfer “in” EC2 instance is free
– Data transfer “out” of EC2 instance is $0.19 per GB for Singapore region
– Bandwidth internal to AWS Region is free for all AWS services except EC2
– Data transfer between EC2 instances in the same Availability Zone is free
– Data transfer between EC2 instance in same region but different availability
zone is charged $0.01 per GB in/out

www.cloudthat.in
Instance Purchasing Options

• On-demand Instances
• Reserved Instances
• Spot Instances

www.cloudthat.in
Elastic IP Addresses

• Allocate and assign IP addresses programmatically

www.cloudthat.in
Elastic IP Address - Pricing

• No cost for Elastic IP addresses while in use


• $0.01 per non-attached Elastic IP address per
complete hour
• $0.00 per Elastic IP address remap – first 100 remaps
/ month
• $0.10 per Elastic IP address remap – additional remap
/ month over 100

www.cloudthat.in
Instance Elastic IP and Public-IP

• Avoid using public IP or elastic IP for internal


communication between your EC2 Instances
– Bandwidth charges
– Lower throughput
– More variability in performance

www.cloudthat.in
CloudWatch

• Pre-Canned Metrics for all services


• Alarm
• Events
• Logs

www.cloudthat.in
CloudWatch Pricing
Amazon CloudWatch Detailed Monitoring for Amazon
EC2 instances (at one-minute frequency)
– $3.50 per instance per month (the per metric price below x 7 pre-defined
metrics per instance)
Amazon CloudWatch Custom Metrics
– $0.50 per metric per month
Amazon CloudWatch Alarms
– $0.10 per alarm per month
Amazon CloudWatch API Requests
– $0.01 per 1,000 Get, List, or Put requests

www.cloudthat.in
Questions?

www.cloudthat.in
Thank You

You might also like