Aws S3: A Quick Introduction To Amazon S3
Aws S3: A Quick Introduction To Amazon S3
Aws S3: A Quick Introduction To Amazon S3
To know more what should be included in the AWS course to learn, check out the
syllabus details first.
How is S3 Secure?
How is S3 Durable?
Data Storage in S3
Storage Classes in S3
Data Organization in S3
Amazon S3 with Example
How is S3 Secure?
Data encryption for the stored data can happen in two ways:
o Client-side Encryption
o Server-side Encryption
Multiple copies of data are maintained to enable the regeneration of data in
case of data corruption.
Also, versioning is performed wherein each edit is archived for potential
retrieval.
How is S3 Durable?
After this discussion, the next question that hits your mind directly is what
kind of data can be stored in AWS S3?
Virtually, almost any type of data can be stored with S3 in different formats. If we
are discussing the capacity, the number of objects, and volume that we can store
with S3 are unlimited. An object is considered the basic storage entity in S3 that
consists of a key, data, and the metadata. So, Data can be divided into two
categories further:
Moving ahead, let us try to understand the Amazon Simple Storage Service with
the help of an example below.
Based on the above parameters, it is pretty clear that N Virginia is just a suitable
region for the Company to host its website because of the low latency and the low
price. Irrespective of your location, you may bid on other regions too that suits your
requirements the best because there is always flexibility to access S3 buckets from
anywhere.
Other than this, there are chances when the website is launched in another region,
and backup is taken in another availability region. This feature is added recently to
the Amazon S3 system and pretty much easy to use. To evaluate your skills, how
much you know about the AWS platform and its technologies, take a quick quiz
first.
Compliance Capability
Flexibility Management
Durability, Scalability, and Availability
Systematic Work Approach
Data Transfer Flexibility
Large Data Storage
Data archiving
Backup & Recovery
Let us start with a brief introduction to each of the features one by one:
Next popular data transfer scheme is Snowball that suggests the interesting idea
of transferring data physically. Here, Amazon sends equipment to premises where
you are free to load the data. It has a kindle attached to it with client’s address
including when it was shipped from Amazon. When the data transfer process is
complete on Snowball, Kindle will change the shipping address back to AWS
headquarters where the Snowball has to be sent.
If you have large batches of data to move, then Snowball is just the perfect choice
to consider. The average turnaround for snowball is five to seven days. In the same
way, Transfer Acceleration can move up to 75 TB of data on a dedicated 1 Gbps
line. It completely depends on the use case and you could take the decision
accordingly. Moving ahead. Let us discuss the overall S3 pricing and how much it
can cost you?
If you are a beginner, then you can start with AWS S3 for free. Once signed up,
new users get 5GB of S# standard storage, 20K Get-Requests, and 2K Put-
Requests, 15GB of total data transfer each month for approximately one year. If
you want to exceed this limit, then there is a certain amount that Amazon charges
you. For this purpose, continue reading this blog ahead.
S3 has plenty of features; still, it is affordable and flexible when it comes to the
payment. It allows you to pay for services that you are actually using. The below
table will give you a better idea of S3 pricing for a specific region.
Here is a quick idea of how cross-region replication is billed:
If you replicate 1K objects, then you have to put requests to store 1000 objects and
inter-region data transfer. Once data replication is complete, the 1000 GB will incur
charges based on the destination region.
These are the fixed service charges that you have to pay. Apart from this, you can
check the website, and mostly charges are given exclusive of shipping days,
shipping days are free. Transfer Acceleration pricing is shown in the following table:
Here, AWS S3 charges are quite manageable and affordable when compared to
its benefits. You just have to understand which package suits you the most as per
your requirements.
Let us understand the concept with the help of a real-time example to assimilate
what we have learned so far. IMDB is a popular internet Movie database to store
details about movies, TV programs, video games, etc. Let us see how did they
exploit the AWS Services to implement this movie database. To get the lowest
possible latency, all possible outputs are calculated in advance with a document
for every combination of letters in search. Each document is further pushed to the
Amazon S3 and thereby to Amazon Cloud Front, and putting documents physically
closer to users. The theoretical number of possible searches that can be calculated
in mind-boggling. For example, a 20-character search has 23 x 1030 combinations.
In practices, by using the IMDB authority on movie and celebrity data, the search
space can be reduced up to 150,000 docs and can be distributed in just a few
hours.
Let us first learn, what is a static website? Have you any idea about it?
A website that only supports HTML, CSS, or JavaScript but server-side scripting is
not supported. So, if you are looking to host a PHP app or Rails app, then you
should opt for any other option. Moving ahead, let us dive deep to the wonderful
world of hosting a website on AWS S3.
1. Here, you have to create a bucket in the first step, navigate to the S3 in the
AWS Management Console and hot the option “Create Bucket.” Here, you
will be asked to give a bucket name and a region. If you want to add some
subdomain, you can use it for the bucket name. For the region name, pick
something nearest to you and hit “Create.” With a matter of instance, you
can see that your bucket starts appearing in the console.
2. Once the bucket is created successfully, it is the right time to verify a
Bucket. Under “All Buckets” option, search for the bucket name you have
created recently. If it appears in the list, it means the first step is completed
successfully.
3. In the next option, you should enable the Static Website Hosting. The only
thing you have to do is, select the option from Properties panel on the
right. The static website can be hosted entirely on Amazon S3. As soon as
your bucket is enabled for the static website hosting, all the content is
accessible through web browsers via Amazon S3 website endpoint for
your bucket.
4. In this step, you have to create an HTML file. Make sure that the index
document is set to index.html. It is possible to set up an error page
according to requirement. When you are done, just hit the “Save” option.
One more nice thing about Aws Management Console is that you could
upload files to your bucket right from the browser. Let us start by creating
the index.htm page. Here is the HTML code for the same.
5. <!doctype html>
6.
7. <html>
8.
9. <head>
10.
11. <title>
12.
13. Hello, S3!
14.
15. </title>
16.
17. <meta name="description" content="My first S3 website">
18.
19. <meta charset="utf-8">
20.
21. </head>
22.
23. <body>
24.
25. <h2>My first S3 website</h2>
26.
27. <p>I can't believe it was that easy!</p>
28.
29. </body>
30.
31. </html>
32. Now, we will learn how to upload a file in a bucket. To upload a file, you
should select the new bucket option. However, it cannot be seen through
the browser, yet everything in AWS S3 is private.
34. Now select the index.htm file from the console and go to the Properties
Tab.
35. In the last step, you just have to verify the result. Paste the URL link on the
browser and check if it is working or not. Yeah! It is that easy creating a
First is Transition action where objects can be moved from one storage
class to another.
Other is Expiration action where objects are removed automatically when
they expire.
It is clear that transition actions are used to manage storage classes for objects
within a bucket while expiration actions are used to delete an object automatically
as soon as it expires. In this way, it is good if you want to store log files in your
application for a certain time period only. When the defined timeframe will expire,
log files will be removed as per the requirement.
Further, lifecycle management helps in moving objects from one of the real-time
storage classes to the GLACIER class. Here are a few interesting facts you should
know before you complete the data transfer process from one storage class to
another.
You can move one object to AWS Glacier, but it cannot be moved back to
the standard library. If you want to do it, you should first restore the object
hen copy the object subsequently with the storage class settings.
It is not possible accessing objects moved from S3 bucket to the Glacier
through Glacier API.
Amazon stores each 8KB data block of information in S3 when you list all
objects in the real-time.
It may take up to 5 hours or more when objects are stored from Glacier as
a temporary copy. The object will retain in Glacier until it is not deleted
from that space.
Encryption in Amazon S3
Encryption means encoding an object in such a way that only authorized users can
decode that particular object. Basically, it is possible protecting the data when it is
passed to Amazon servers and while the data is stored to the Amazon. Also, to
protect the data during transmission, you can use SSL (Security Socket Layer) to
transfer HTTP requests. With the help of Amazon’s Java SDK, it is possible to set
up the protocol using “ClientConfiguration.” Here is an example for your reference:
clientConf.setProtocol(Protocol.HTTPS);
return AmazonS3ClientBuilder.standard()
.withClientConfiguration(clientConf)
.withCredentials(credentialsProvider)
.withRegion(regionProvider.getRegion())
.build();
Data encryption for the stored data can happen in two ways: Client-side Encryption,
and Server-side Encryption. Multiple copies of data are maintained to enable the
regeneration of data in case of data corruption. Also, versioning is performed
wherein each edit is archived for potential retrieval. In the case of server-side
encryption, data is encoded before it is saved to the disc. Your client application
managed the encryption process and sends the data that is already encrypted to
Amazon. The best thing about server-side encryption is that Amazon has already
encrypted the data and performed the key management for you.
Data will be in its original form for the moment while it is stored on Amazon
machines. With the client-side encryption, you can encrypt your own application
and learn to manage keys. This is the way how data is protected in the memory of
your client’s machine and Amazon never sees the original content. Here, you have
to focus on key algorithms and key management yourself.
Amazon offers three different types of Server-Side and client-side
Encryption:
Further, a unique key is also encrypted with a master key that rotates regularly.
This service uses Advanced Encryption standard for encrypting the data. Make
sure that this encryption technique encodes only the object data, not the metadata.
This type of encryption offers key management services that are designed to scale
for large distributed apps. Here, the master key is created differently when
compared to the SSE-S3 encryption process. However, both processes allow
creating keys, define policies for them, and audit logs for the usage of each key.
This is the default server-side encryption scheme used by the Amazon. The key
can be used in subsequent calls unless you don’t specify another key.
If you don’t like the idea that Amazon provides the key for data encryption, you
have the flexibility to define your own key that is customer-centric key as the part
of your requests. Further, Amazon implements the same key to encrypt or decrypt
the data on the server-side. Amazon stores the HMAC value of the user-created
key to validate future requests. In case, you lost the key created by your then
HMAC value can help you to retrieve the same key and helps in robust key
management.
Here, the client generates a random key to encrypt the data before the final upload.
It then encrypts the data with a master key suggested by the client application and
sends the encrypted data with a master key to Amazon. In this way, Amazon does
have any idea about the raw data and the key used by the client for data encryption.
Versioning
To protect your data, from unwanted deletions or overwrites, you could enable
versioning for a bucket. It will help in creating a new version of the object and
upload the same version instead of overwriting the old data. Versioning is enabled
on a complete bucket instead of any single objects. Once versioning is enabled,
you cannot disable, but you can suspend it.
Final Words:
In this blog for AWS S3 Tutorial, we learned everything about the storage service
from the basics and how to set up a static website using Amazon S3. This guide
discussed the features, benefits, and usage of the service. We also learned
different encryption mechanisms that can be implemented on the raw data as per
the requirement.
At the final leg of AWS S3 Tutorial, we will recommend you joining the AWS
certification program online at JanBask Training and know everything in depth from
the beginning. We wish you luck for a successful career in the AWS space. All the
Best!