By the end of the course!!!
- Types of cloud storages. Object storage , block , File storage
- Object storage ( Definition , examples of object storage, properties of object storage,
storage classes on s3, how to use s3 , Security measures we can implement on s3)
========>> Cloud storage===>>
In the cloud there are types of storages that can be used depending on the purpose:
- Block storage: With this storage data data is stored in the form of blocks in equal
sizes and with each block having a unique [Link] is the smallest form of cloud
storage.. Block storage is used by virtual machines.
Examples of block storage include :
- EBS Volume ad instance store in AWS
- Azure disk storage in azure.
- File storage: With this storage data is stored in the form of files in a hierarchical manner.
File storage is a form of shared storage between virtual machines. It supports NFS,SMB
etc. File storages are persistent storages and VMs can be mounted to these storages
through a mountpoint.
Examples of File storage:
- In aws we have services like EFS, FSX and FSX lustre
- In azure we have an azure storage class where we can get file storage..
- Object Storage:
With this storage we can store data of any type. It is the cheapest and unlimited storage
type. Any data stored in object storage is called an Object. You do not necessarily need to
mount a VM to object storage in order to retrieve data from object storage.
Examples of object storage : In AWS, AWS S3 , IN GCP GCS , Azure BLOB STORAGE
OBJECT STORAGE: AWS S3
AWS S3 STANDS FOR SIMPLE STORAGE SERVICE. This is an unlimited object storage
service that is highly available and durable. The s3 service is region specific.
In s3 , we will store our objects into a resource called a resource bucket..
USE CASES FOR S3:
- For storage, backups of existing systems etc.
- They might want to store logs..
- Used for datawarehousing ….
- For hosting the frontend of an application ( Website ) ……
● S3 buckets:
○ A resource created from the s3 service which is used to store objects.
○ An s3 bucket is highly available and durable in the sense that:
■ Availability: Data stored on s3 would always be available when needed.
S3 provides 99.99% availability of data stored in s3 every year.
■ Durability : When data is stored in s3 , the chance of losing the data is
very low. AWS provides 99.999999999% durability.
○ S3 bucket names are globally unique.
○ In an aws account , the max number of buckets is 100
○ A bucket is unlimited , that means we can store as much as we want to but an
object size is limited to a max of 5TB.
○ With s3, WE CAN file into a bucket , Anything we upload into an s3 bucket is
called an Object.
○ Every bucket has an arn (Amazon resource name):
arn:aws:s3:bucket:bucketname
○ Objects uploaded into s3 would have :
■ key (name of the object)
■ Size
■ S3 URI: this indicates the path taken to access that object within the
bucket.
■ arn : mostly gonna be the arnofbuck plus object name
■ Object URL (uniform resource locator): This provides the link to
access this object from the internet.
■ Every object must belong to a storage class
■ An object can be deleted from the bucket.
○ With s3 buckets objects can be accessed over the internet through the Object
URL. By default this is not possible because s3 buckets are private by default
hence all objects are [Link] order to make objects public:
■ Firstly, turn off block public access.
■ Secondly, you can make the object public in two of the following ways,
● Throw ACL
● Bucket policy.
● NB: ACL should never be used to control access to an object.
Bucket policy is highly recommended in controlling access to
buckets and its objects.
● Storage class:
In an s3 bucket data must be stored in a storage. A storage class would determine the
cost, access pattern, availability and durability of an object stored in an s3 bucket.
○ Standard: This is the storage that is meant for objects that we want to access
frequently. It is the most expensive storage class and it provides the best
availability % (99.99) as well as durability.
○ Standard IA ( infrequent access): It is a storage class meant for objects that we
don't access frequently but when needed we want to retrieve them very fast.
○ One Zone Infrequent access: It is a storage class meant for objects that we
don't access frequently but when needed we want to retrieve them very fast.
When an object is uploaded to this class , note that it would be available to
just a single availability zone.
○ Intelligent Tiering: This is a storage class built on machine learning. With
this SC, it is good for objects that we do not know their access patterns.
Intelligent tiring has the ability to monitor objects and determine how to
move them from one subclass to the other. It has 3 sub classes , Standard ,
IA and Archive. Intelligent would automatically move objects from one subclass to
the other depending on the access pattern to help save on cost and would also
ensure no performance degradation for applications trying to access the objects
from the bucket.
Archive classes:
With this class , it means to store archive data. Archive data is data that is not needed all
the time but is kept for regulatory and compliance purposes.
○ S3 Glacier:
This is a storage that is good for storing archive data and its retrieval period would
range from mins to hours depending on the type of retrieval.
○ S3 Glacier deep Archive:
This is a storage class that deals with archive data that is mostly needed once or twice a
year and its retrieval time is more than 12 hours.
● Versioning: This is a feature that prevents unintentional delete of an object in a bucket.
Versioning would ensure we can have multiple versions of the same object.
○ When versioning is not enabled in a bucket:
■ We cannot have objects with the same name , the new object would
delete or overwrite the existing object that has the same name.
○ Versioning can be enabled or suspended.
■ When versioning is enabled :
● Objects with the same name could exist in the bucket but as
different versions of the object
○ If we want to make use of versioning , we simply need to enable it.
○ If you delete a version object :
■ The object would not be deleted , instead a delete marker would be
placed on the object. A delete marker is a pointer placed on a versioned
object , which means a request has been made for an object version to be
deleted.
■ You can only delete a versioned object if a delete marker has been placed
on it. When a delete marker is placed , you can choose which object
version you want to delete.
■ Object versions are unique as they would have their own object
properties.
● Hosting a static website on s3:
○ S3 buckets can act as a medium for hosting the frontend of a web based
application. It is the cheapest way of hosting a front end (Website) on AWS.
■ A website is used to display web-pages for end users to interact with an
application. The web pages are written in HTML.
■ If you want an s3 bucket to be used for hosting a website , you have to
enable static website hosting.
■ LETS HOST A WEBSITE ON AN S3 BUCKET:
● Create the bucket and enable static website hosting.
● Upload the html file containing the code for the website
frontend.
● Make sure the bucket is public and the objects serving the
website are public.
● Through the website url , we should be able to access the
website.
Assignment: Identify any security misconfiguration that we did with the above website hosting
● Encryption:
○ ENCRYPTION AT REST: Encrypting data while it's not moving.
■ There are two types of encryption at rest on s3:
● Client side encryption: This is a kind of encryption done on an
object before the client uploads the object to the bucket.
● Server Side encryption (SSE) : This encryption is done at the
level of s3.
○ SSE-S3: SERVER SIDE ENCRYPTION WITH S3 Is
completely handled by aws. AWS generates the encryption
key and uses the encryption key to encrypt your object
before storing it on a hardware server. It is the default sse
in aws s3.
○ SSE-KMS: with this , you create an encryption from aws
kms service , you provide the key to s3 , anytime an object
is uploaded to the bucket , s3 uses the key to encrypt the
object. Sse-kms comes extra cost ..u are charged for every
encryption and decryption.
○ SSE-C: server side with customer provided keys: You
would create your own encryption key, provide that to aws
s3 , and it would be used to encrypt your objects while at
rest in the bucket.
● Server access Logging: Server access logging records any activity an identity performs
on an s3 bucket. When an activity occurs it is recorded as an event. The events get
stored in a target bucket.
● Event Notification: It is a feature on s3 that would keep track of on the occurrence of a
particular event and when it occurs it sends a notification to either of the 3 destination
services : AWS SNS , AWS SQS , OR LAMBDA.
○ CASE STUDY: NOTIFY US THROUGH EMAIL WHEN AN OBJECT IS
DELETED FROM ASHY-APPLICATION-BUCKET (Assignment)
■ Enable event notification on the bucket for every object delete
action:
■ Create an sns topic and subscribe our email to the topic
■ Connect the sns topic to the s3 bucket.
● Object Lock:
Object lock adheres to WORM models…Write once read many. It is a feature used for
compliance purposes in s3 where objects in a bucket would be prevented from being
deleted over a period of time ( Retention period).
Compliance and the governance mode
● S3 replication: This is a feature that, when enabled , would ensure every object
uploaded to a bucket (source bucket) , a copy of it is made available in another bucket
bucket ( destination bucket). The destination bucket can be in the same account , the
same region , different account or different region.
When the destination bucket is in another region , it is called CRR ( cross region s3
replication) .
When the destination bucket is in the same region , it is called SRR( Same region s3
replication) .
Replication of an object in the source bucket would only happen if the object existed
after replication was enabled.
In the case where we want already existing objects to be replicated, we can use either
s3 batch copy or AWS data sync
The source bucket needs a role with permissions to put objects into the
destination bucket.
● Presigned URL: This is a feature on s3 objects that is used to provide temporal access
to a private object for a period of time. After the time expires we cant access the object
anymore.
### How do you save on cost in s3 :
Lifecycle Rules:
This is a feature that is used to automatically transition objects across storage classes
and also expire objects after a certain time period ( days)
Prefix:
The name between the bucket name and the object name…..
Folder: A folder in s3 is simply used to organize objects in a bucket.
Imagine we have a bucket called schoolrecords and we want rto store records for every
class ..in the bucket we can have a folder for each and every class.
s3://xashy-cloudsecurity-records-folder/batch-1/aws/[Link]
s3://mybucketxashysource/[Link]
s3://mybucketxashysource/pictures/google_logo.png