CCSM Unit 3

Download as pdf or txt
Download as pdf or txt
You are on page 1of 16

pro gachin (eerms

Date Saath
Unit-&
Short note on EC2S2,:Coogle App Engine.
{Gopgie App) Engin cQAED
Schedu(ar
chubby

Applicaton Google
loud Biq Table
inastru.
Server
Node
Iuse
|Map Reduce
NOde job

Schedular qFS
Slare
chunkserver

Lin Ux

Store mplements transactions


The data
its distubuted n etwUorK using entity
acrOSS
qroup".
Gon gle App Engine is the ty plcal eKample of
Paos.
deueloplng and
aoogle App Engine is for and theseproCeSS
hosting
applications
web
Qrehiqhs Sealable.
A Hranscations mantpulates etites within a
single gxoup
theSame qroup are
Fege Nc.
stored
i
Saathi)
Date

Hogether fox effcent execution of roosotors


pxoviders Can uSe
Third parts applt cations
GAE o build cloud applcations for prouiding
Seyices
Theperfoman co o f t h e d ta sfoxeDCaA

be enhanced by in- memory cching Using


memcache cshicb can be used
the
indepen dents of the data Store,
An apDli cation
Goo gle AccoUnts
user qothentcaton

GAE providexs the qbility to mqnl pulate


imageg
Lmage data usiDg the dedicoted
Sexuice Ohich otate,
car esize, fip: Crop
and enhance im aqe
Conne eh'on Can
The Google sDC Se
SecuYe b ate
tunne though the Intern et an d link your
0Y
intranet to a n external CAAE application.
l The URL fetch Qperatfoo pxovides the abilty
fox appli cations fetch gesowxCes
Cemmunicate with otther hosts Querthe
nternet cusioq HITP qnd HITPS quests
aceeSS. yesoOTees On the
Applicqttons Can
(ntern et, sycn as web seruices othen
dota using GAE'S ORL fetch Service
FeatoYes of Qoogle App Eogine
UsesS build the application using
|anguaqe YUNtmes Such as Taya Pyfhon
Ct#t* Fahy, PHe efc,

CUstom yunimes allow


USesS xOy ibrary and AanelorIs
Date
Saath)
to App Engine by supplylng DocK er coDtainer.

3. Powerful applcatton diagnostCs


Qoogle Agp Eogine Uses cioud monitoing
loqging too monto the heatth
qnd cloud logging
perfor mance of the app and to dia gonse
fx bugs quickly
Appiication vers bning
easilus hosts dieferent veTsions of
the app, and Create develop ment testing,
stagiog Qnd pxo duction en ulyonment.

Yoyr
appll cation perform these tasks
sch edule tht youconfqre Such Qs
On daily or hourly basis usingCon johs"
Seryice.
handled by the c

(2) Ec2 (Eastic Compute clOU)


Amazon as the first company to iotrOduce
YMS in appll ction hosting.
Custoners VMS instead of physical
machinesto UO their appli cations
-
Jsing VMScUstomerS |oadany
SofwgTe of thelr chofce.
ela Sti c feqtne Such a SeTulce
The
iS th at a CUStomer CYeate, launch,
and Aermioate Seruer lostan ces as need ed,
paying by the hour for active SOrverS.

Amazon provides seueral types of prs


insialed vns nd its (nstances a
oftro called Amzon t4achine lmae CAMs),
SaathD

aachine lmoge
Date CArnazoni
AML pzivat
Types of axe
ohtch tootter
. rivate AM you,access
created by
get aceSSto
dmages Qet
ou can images.
pxivQte
pYiVQfe
bu defaut lqon ch qoux
users to
o
o0d1elea sed
a. Pblic A l users can laonch
Created by
aOyoneond USe them
tna ages community So
the Atls based Iists a public
instaNces ( k e . Aus
heq
ang CuQy
images at" specific
AML CTeate maqes pouidiog
Can
cao -be
launcned by hour
fen ctons hct per each
oillinq. to pay sou
anyone top of Amzon's chorqes.
of usage
templates for ioStane
AMls Yunninq vS.
whicb
to
he Teate key Pair
Cseqte qn AM ’
Confiquxe EireoalI ’ Lqonch
OSer to Con fiquse their
Tt qlso q(ous the
their nequlrements, ie
Instances a s per
Qllocate the
RAMRom, Qnd stoyage
the need CUYYent fQsk
accoTding
PooUide qoo to ent VItual COmputers -
prouision f Seruers Cloud
|The
is one the eaisest
EC2 hosYesizable Copacts, C2 ofrs
ty,
SecuTity, ellabitity htqh- peY fomance
pro- gochi (emmha
Date
SaathL)
Cost-efe ctlVenfastructuxeso Qs -to
meet the busioess néeds

Ama|on S3 pouides simple webSenIces


interfa ce th gt Can be used to Stoxe and
TetyieUe Qns mout of data at ans time
fom anywnere the web.
provides the obiect -oxlented stoxa ge
seruice : for Users
User s Can ess thelr objects thxough Simple
Obiect ACcess Protocol CSOAP) (oith
with eorow sers
pro grams ohich Supports SOAf.
The. fundamental op eration unit of S3 is called
an object ond
Each object s Stoxed i n a buckket
getrieU ed ung,ue, deueloper -assigned ke4
Bucket is the container Of the object.
Besides uni que key atributes
attributesthe objec
has other attributes SUch as Values
metadata, and access control infomattoo.

Amazon S3

liI Stomnelasses?
Amazon S3 offers Q TeAge of Storage daSer
desiqned for diferent uSe Cases
Store mission -crtical
2. For example you
producti'on data S3 standard fos frequent
accesS, qve costs byStandavd
Storing inhequentty
-TA G SA one
access ecd data io S3 S3
Zone-IA,Qnd achfve data n the lowest Costs
Fzçe Nc.i
Date Saath
in s3 Gla cey Instat Retievot S3Glacier
AeribIe Retvtevaland S3 GLa cier Deeo vchive.
manaq em ent
2.Storoge
Am Qzo0 Se has StorQge manaqemnt
featux es that you can use to manage
Costs, neet Yequlatory Teq uirements reduce
lattncs ond Save muttple disttoct copies
of qouY da ta foY Compilan ce requremer
S3 Lifecycle- Congiurea ifecycle policy to
manage youx ohjects and Store th th em
COSt eetectivels thraughout thele Ifecycle

S3 Objec Lock- Prevent Arnazon Sa ohjects


from being deleted or
Or Overwritteo for a
Gxed amount of ttme r Inde finlkls
3 Reolicatfon- Replicate objects their gespectv
metadata object tags to or Mor

destinatron buckefs the Same Y

dferet Aats Reg ions fox educed


lotency com pliance Securits ather use
cases

S3 Batch opesatlens- Manage billons ofobject


SCale wltn single S3 API equeste Censole
clicks In the Arnazon S3

hovides ffaluses -for Qudting


QccesS o qour buckets anc
and mn Gnaging
cbjects:
pro- achin onte

Saath)
Date

Bu delault, s3 buckets theobËects.


in tem Teprlvate.
hrant qranule YesouxCe permission s that
R To
Supporf your specifc use case os to Qudlt
s3 eSOUYCe,
he permìssion of Qour Amozo0
features ke S3 Block Pabiic
Amazon S3 access polnts, Bucket
ACcesS,
polictes, Acess contol l|st, Access Aooyzer
fr Sa etc.

4.D0ta procesn tigger wD0kfoos


to qutomate a Variets of otner processing9
can USethe
Qcivites at Scale, qb0
followg featuyG.
to S3
s3 o bj ect Lambda- Add souY OOn Code
GET Yequests to modify proCeSr date
as it is eturned to applicgtion

iqqer wokous thqt


2. Event nottfcation S-
USe Anazon Simple Notifcatlon: Seruice CAmQzon
SHS) Amazon Simple Queue CAmQzon SQs.
to change ls made
AlS Lambda
to youSS S3 eSourees.

5 Storage
AmQzon s3 provides loqging manitovlnq
tools that you USe o montor
Contol how your Anazon S3 esouTEeS
USed
being
1 Automafd monitorino tools
Date Saath)
Amazon ccioud gtch meica foAmazon s3.
Track opErationgl heathof youY S3
esousces Con Rgune blllnq Qlerts. when
etinated chaxges Yeach us er-detoed
threshold.
Record actions taken by a cSen,aoe

Q Manuol monitoriag tools


SerUer Qccess loggrog - Qet detaled Te Cord s
for the geqets that
bucket
AhS TUSted Advisox- Éualutate your Qccaunt
by using Akls best p ce checks to ideatf!
as to optnize yauruour AS 0nGastructure
Imroves Seurts god perfozmonce redoce
costs and montor Sexvice

5 Analgtcs and insights


|Anaz00S ofkIS teatunes to help bettcr
unders tand anatyze, optimizg your
at SCale
i Storage
Amgzon Sa
Storqqe ens- Cndexstands analuze
andoptonize your storage. t pxouide 29+
intera ctie
usege and actuity meicg
dashboarós tD qggeg yQte date to
Specilic Qconts.

2Stoyage Cass Analysis- Analqze storgge acces


þaity0Sto decide uhen itS_tme to Moe
to e cost - effctive StNage cqsS
prt- hin amte
Date
Saath
4MicxOSoft AZune
MMCOSOft Azefonmerly koonn as
iodoos Azue , Is micxoso4S publfc coud
ComputiOg pletform.
2. It pxovides a broad mange of coud
Serwicesin clodiag compte andlyticS Stoage
netuooki0Q.
g Tne Azuxe platox m oms
cims to helo
busslnessesm manage challenges. Qnd meet
eir Örganlzation goal
4 This qives users a feKibility to use
qlue
their preffexed tools and technologies.
5. An additton,
o-ARrS: four di£'exent!
forcns Of cloUd computtoglà0s Paas, Saqs,
and SeUeess funcion s.

Each categoy incude numneroUS


#\nstance seruice type Most populgr
Sesvice
ComHe- hese seruices enable user to
deploy and manage VMS, contalners
batch jobS. os oel as Suppoxt gemote
applfcatton acceSS
2Moblle- These pyoducts help deu elops
boild cloud Qopicatton tor moble deutces.
prouiding nottficattoo geruices tools for
building gppltcation program lnttrface CApl's)
iQ abilíty to Contt with data
2leb- These SeyuCeS Suppo1S the
deu elop ment
4Storaas- This catsqoxy of services
prouides Scalable cloudStoxage far
Façe Nc. i
forineal-tBmesensorg QAE
data:'Qnatytica
analsttcs,big'
captuTe, &
explan for Stem
Cag
betm.
Qs
huqe
oeiotcrneth
leasniDg, 0f ch seueral
qssumptions
concorinq
seYvice
sy ComputrssU SOch
system
file compotfog GFS,Of
as amaunts
tems Ioterfae
Sesuers num
oom
0seTS fle hold in
aswell
details si size
Qqe distruboted
data
datQ massive
unzeliqble
CheapQ
tile Cleqy coud of
nelÙ Stox S filelaxq
Qtorgge functton
avChiteetoe
/n fundamental-buted failures
LoT
ToT engine.
Search a o thea
SeruiCes GFSexplain
:GESexplain
sgahat store
mochiOeanaluze
angtyze tru
be
should
the Ql hold
velated tme
the Conc
deuiceg oS
These dis lnteyfaQ
PoSLX ConcoYeA
Qnd needed
Google
yedundanty will
(Tor).
thinqs
These
TOT-
8.
amount
qe
lax
raditional applrcations
5.There
aYe.
5.
An
alytics akes 0sa depYOVi
not is QFS
Hu
data
analyttas Q er aw
2the
oth
monltor s on
data There
4. oneharduQxe
G.An
Google's
Same hat
and Tt
to
Date ite
k
prot sahin onhe

Date
Saathi)
and adoptcd. Someme thods to. achfeve
this goal
4 Write a S:N apen SoUTCe tools fo
buildinq priuate cloud
cOVd computng gives yoy the abilits to
leNeTQgc. both openSoUY Ce and closed
Source 4echnolog4 demandwhich tuns
Hhe RO) mode 0pen SoUrCe
for
These t0ols ce lag S clouds,
Ëncude open source
Such Os 0penStack anc cloudstack th at
many use wth
wth publlc ciovds to cYeate
DIY cloud man ag ement that ts prlvate -
clovd- cenhed.
The four oDen soUTCe cloud managemert
ola+forms that fbllow all do coud
management au have the
di£ferentts, ht al
Same abiectire
C Apache cloud stack
Cloudstack was ioitlally pitched gs an opn
SouT Ce laas cloud platform far both
private and publie cloud s, but claue is
today t Is a tue cloud management
platfortn
The Softwaresupports the qeatetno
of hypvisiorS 0fany taol out there
lcurYentu
Tne istofsupportd hyperuisiors
IncludesVMware,kuMCLAYIX, Xen Co ud
PlatfoYM0racle vM sevel microSoft
IHyper-J
Cloudstack aiso pxouidean APl hatls.
j2 IN Thelb.
iient
e g.
Server.facilites Cluste Date
The Cochmaster
focd
r Ahe Thedirecty
between A0 Qremoltiple
Ihere isa Eile(00MB
Hles,
cach

mot control the ppended.


9ettfng
aton mutetto0 ie Qnd writttn one
4Pimary data
condary Secondar Ciient
asksthe gpiica 3Se messages Systtm t block
esthe plíca futuxe
use. SingleSin
seuence the
manaqedmetadrta.
peraions Stovethe qle
size
Cientthe
Stp
oamespace
transferred
between
clients ave master
hunk
ch qftr size
sulte
lloeing in b 6a
in
Legend : Master Qnd performed Qre all MBfe3 that
wth
ufS. chonk
tqndhe the Qnd the
Dqta in
th the
COmmon.
Stea
StepS ey m@ster.
lockinq Lohde
d
Saath)
can data
of
be 64t8
the that
yeplielicas. write
a eTOTS
Any
when nep
al SQmepimary
sevia by an
Ose
be
operation
pYovided
Saath
0then
ach
OTder. the to end
fa
Ho
ofcient
the te
dthe Tequest
wrjte
the
secondlaTy
Gach pmax4
fhe
of(ocattons
motatonS.i
-only
Qgain
edqed
qcknul
bave
sends t
mepos
axe
d
entty ted
Comple
have
the
to
reply cient to
the
he:sld to
data
qta
d
in the
the
the Fach
clien+ eplicas.
Secondqry- sOperqtion
locationUnreahqble UntI
.of
s- cQche fbe hig
blockS
future
Coitamastera(ease
ho(ds theany bs
mutations
applies eplicas
ny witiog
the
clieotpushes psimgry. assigned to
andthefosContQct:thesto out
Qged forwgdsall ey
twlll theTpiicQs xeplies
thatth QFSdata
the
appending
Special
Steroeples
ma becomes
Primamyanathe
ay
prin
SO
do ed
de
data buffer secondaxy'3 the
the order
fonger
no SeYuer primary
The
s primay files
the
LRO to besides
hus
chonk this operation
encountred append
iodicating to
eceivng
Tequest number
A lnternal
Cast,
epli to
Deeds Used
es
Cach
Cttent plíca
Chonk once
4
the he the
Date
The Ahe The 64The
Saathi
Date

Cormpatble
organi2 QtBons that ant to deplos hybrd
Clouds.
cloudstack s the pnlvate cloudAWS
is the publtc cloud.

a Openstack ao laaS
Ake clou dstack open stack. is
prN Qte on publlc cloud.
Conhol:a laxge pool of compute,
Storaqe aod efoork esouT CeS eith ee
wlthia a dcta cotre asas aprivate.
a clou d.
Openstack iwor ks: Q plous well with pbpulak
entrerrise dpeo souTCe technologies,
mgkin g it ideal for both Open stack
n exttnal hublic cloods
HYloreover, Openstack has more nteres
Cloudstack, fom a lasqer po pulaton
of deUelopers that Suppoyt the bQse code
Q mu ticloud
|deploymeri Rom the open Stack private
cloud Using nattve Open - SouTCe
open S Cloud
manaqenent tools

anaael la. is an apen-SoUYCe nanagement


platfbxm for hybrid IL meanlng for a
6f leaacy and public &'pxiVQk louos
mla óf legacy
Thetools Can manae Smal laNge
enuiconments, Qs well asSupport multinle
SUch Qsvlrtual machin es
technalaaes
public clouds Qcantainess
Date
Saathi
The mangge IQ too| aloos (gou to
Understand the coYYent State of our
enuironment xOvide self- sexuice tox end

Performonceoptimiz ation ant utlizgtion qne


carried Out siAg mnanagemeat and
monitoog
Cloudifu
-dsiven
Cloudifu lsQn apen - souTCe, nodel
diectedgt
Cloud manggemert tool thatls
Lmult-cloud incudinq autbmato
HobsQ ctibn
Clou difs is desiqned to autbmate he
deploynet, configuation and Temediation.
n etwork seruices
of appli cation aod
acTOSs heteroqenOUS clouds.
USes an Qpproo ch based on the
To SCA(Topoloq4 and Orchestraton
Bpececation for cloud Appli cations)
Specfcatans qowerned by the
Cao be
production team and deployed to
productton.

Nite a uay of deploying web seruiceg


Gom (n side Q OUtside cloud axchite cture
Same as onit |. Questton-t.
Coeploymert mod els)
Saathi so
it paralle
whtch_makeg
combme.. providaq
model
programmlag
a to înto
network
clusttY
distxubuted
a (s ail
Phose.
diuided
m then Hadoup
if:for power
pOcersnq
the
piocessing then Reduce
USe. and tasks mamls hadoop
tnap
Reduce
in
efetcient
ond
to and
Solit mapReduce &
Architecture
Hadoop
eguiualent
obs the
ovex
oyerheQd taste
s Phase
dota~sets
HØFS
and clent irst tQsk
the
etG Mp
andentsOf
Compon
is
Reduce
Map to of
the
produceof ce
S
to MopRedu Tob
puxpose edce
6.Map
Redvce
Reduce
Map fox each
t ph0ses
Crent
data
The
pOwerRl
manneY. xeduce
used Map
he lessto Tnetoo
Date do

You might also like