Get the Most, from the Best!!

7
1

1
 Get the Most, from the Best!!

Topics
 Tagging
 Cost Reduction Opportunities
in the Cloud
 Cost Monitoring and Billing
Alarms
 AWS Trusted Advisor
Get the Most, from the Best!!
 Get the Most, from the Best!!

 Tags are descriptive metadata attached to AWS

resources
 Some tags are built-in and cannot be removed
◦ All other tags are user-supplied
 Use to identify and categorize resources

So, to begin – what are tags? They have a few important characteristics:
• Tags are key/value pairs that can be attached to AWS resources.
• Tags are metadata – that means that they don’t actually do anything, they’re
purely for labeling purposes.
• And finally, tags are (sometimes) inherited. Some services such as Auto Scaling,
AWS CloudFormation, and AWS Elastic Beanstalk can create other resources such
as Amazon RDS or Amazon EC2 instances. Generally, whenever one of these
services creates a resource, it will tag that resource with a reference to itself.
Unlike built-in AWS tags, these tags do count toward your total tag limit for a
resource.
 Get the Most, from the Best!!

◦ Environment (production, test)

◦ Application Environment =
Production
◦ Owner
◦ Department
Cost Center =
◦ Cost Center Marketing
◦ Purpose
◦ Stack Application =
Promotions
 Get the Most, from the Best!!

 The “required-tags” is a managed rule by AWS Config

Parameters
tag1Key Key of the required tag.

Optional value of the required tag.

tag1Value
Separate multiple values with commas.
 Specify the required tag key (and optionally the required
value)
 Noncompliant resources will be identified shortly after
evaluation

AWS Config provides AWS managed rules, which are predefined, customizable rules
that AWS Config uses to evaluate whether your AWS resources comply with common
best practices. You can customize the behavior of a managed rule to suit your needs.
For example, you could use a managed rule to quickly start assessing whether specific
tags are applied to your resources via the “required-tags” rule. After you activate a
rule, AWS Config compares your resources to the conditions of the rule. After this
initial evaluation, AWS Config continues to run evaluations each time one is triggered
and periodically based on a customized time value.

For more information, see

http://docs.aws.amazon.com/config/latest/developerguide/required-tags.html.
 Get the Most, from the Best!!

 Create tag
export TIMESTAMP=`date`

aws ec2 create-tags --resources i-1234567890abcdef0

--tags "Key=SecurityCheck,Value=$TIMESTAMP"
 Query & filter based on tag

aws ec2 describe-instances

--filters "Name=tag-key,Values=SecurityCheck"
--query "Reservations[].Instances[].[InstanceId,Tags[?Key=='SecurityCheck'].Value]"
 Get the Most, from the Best!!

 Start up/shut down instances with a specific tag

simultaneously
 "Tag or terminate" (Conformity Monkey)
 Pseudocode:

instances = describe-instances
for each instance in instances
if !instance.tags.member_of("Required_Tag") then
aws ec2 terminate-instance (instance)
end if
end for

Two commonly used strategies with tagging include using a tag to shut down and restart all
instances with a specific tag, and "tag or terminate."
The "tag or terminate" strategy is shown in this slide in pseudocode. In this scenario, a
company or division issues a set of policies regarding what tags must be placed on running
resources. A script periodically examines all instances running under an AWS account, and
checks that these required tags exist. If they don't, the instance is terminated as being non-
compliant.

In practice, companies that implement this strategy usually stagger deployment over
several weeks. In Phase 1, machines are not shut down at first; instead, the "tag or
terminate" script is written so that it emails the IAM user who created the instance, and
warns them that their instance may soon be shut down due to non-compliance with
corporate policies. In Phase 2 of the rollout, instances are actually shut down and an
explanation of the shut down is sent to the IAM user who created the resource.

"Tag or terminate" is part of a larger overall cost regulation strategy. After instances are
properly tagged to describe their role and function within an organization, companies can
create other automated processes that implement company-wide cost saving strategies -
e.g., shutting down all development instances during weekends and holidays.

For more information about Conformity Monkey, see

http://techblog.netflix.com/2013/05/conformity-monkey-keeping-your-cloud.html.
 Get the Most, from the Best!!

Write IAM policies that enforce the use of specific tags.

{ "Effect": "Allow",
"Action": "ec2:CreateVolume",
"Resource": "arn:aws:ec2:us-east-1:123456789012:volume/*",
"Condition":
{ "StringEquals":
{ "aws:RequestTag/costcenter": "115",
"aws:RequestTag/stack": "prod"
},
"ForAllValues:StringEquals":
{ "aws:TagKeys": ["costcenter","stack"]
}
}
}

You have the ability to specify tags for Amazon EC2 instances and Amazon EBS
volumes as part of the API call that creates the resources (if the call creates both
instances and volumes, you can specify distinct tags for the instance and for each
volume). The resource creation and the tagging are performed atomically; both must
succeed in order for the operation (RunInstances, CreateVolume, and other functions
that create resources) to succeed. You do not need to build tagging scripts that run
after instances or volumes have been created.

You can also write IAM policies that enforce the use of specific tags. For example, you
could write a policy that blocks the deletion of tags named ”Owner” or ”Account”. Or
you could write a “Deny” policy that disallows the creation of new tags for specific
existing resources. You could also use an IAM policy to enforce the use
of ”Department” and ”CostCenter” tags to help you achieve more accurate cost
allocation reporting. IAM policies that mandate the use of encryption for any EBS
boot or data volumes created may also be written. You can use this to comply with
regulatory requirements, enforce enterprise security policies, and to protect your
data in compliance with applicable auditing requirements.

For more information about IAM conditions, see

https://aws.amazon.com/blogs/aws/new-tag-ec2-instances-ebs-volumes-on-
creation/.
Get the Most, from the Best!!
 Get the Most, from the Best!!

 Pay only for what you need, when you need it

 Create scripts/templates to shut down environments
 Unused resources can be turned off
◦ Specific services after business hours, during holidays
◦ Dev/test environments
◦ Disaster recover environments
◦ Tag “temporary” instances

• Principle of cloud spending: pay only for what you

need, when you need it.
• Create scripts or templates that can easily spin up and shut down entire
environments.
• Unused resources can be turned off.
Dev/test environments can be shut down at project termination.
Shut down specific services after business hours, during holidays.
Disaster recovery environments can remain inactive until you need them.
Tag "temporary" instances that can be turned off after a specified time period.
 Get the Most, from the Best!!

 View the overall status of your costs/usage

 Access your monthly bill
 Set custom cost and usage budgets
 Access comprehensive data and reports

AWS Cost Management provides tools to help you access, organize, understand,
control and optimize your AWS costs usage. You can view the overall status of your
AWS costs and usage with the AWS Billing Dashboard. You can access your monthly
bill with the bills page which gives you access to the most up-to-date information on
your costs and usage, including your monthly bill and detailed breakdown of the AWS
services you are using. You can set custom cost and usage budgets with AWS Budgets
which lets you set custom cost and usage budgets that alert you when those
thresholds are exceeded. Finally, you can use the Cost & Usage Report to access the
most granular data about your AWS costs and usage.
 Get the Most, from the Best!!

 “Right-size” instances
 Consider T2 instances for workloads that occasionally require
to burst to full core performance
 Consider purchasing Reserved Instances (RI) for groups of
long-running instances
 Batch processing jobs can be run in parallel and shut down
when work is done
◦ Leverage spot instances to get the best price
◦ Consider AWS Lambda
 AWS Trusted Advisor provides real-time cost optimization
guidance

• “Right-size” instances.
Check defaults for internal tools, scripts, templates.
Is a tool defaulting to a Large instance when a Medium would suffice?
• Consider purchasing Reserved Instances (RI) for groups of long-running instances.
• Batch processing jobs can be run in parallel and shut down when work is done.
Leverage spot instances to get the best price.
Use Spot Instance history reports to fine-tune bid requests.

T2 instances are designed to provide a baseline level of CPU performance with the
ability to burst to a higher level when required by your workload. T2 instances are
well suited for a wide range of general-purpose applications like microservices, low-
latency interactive applications, small and medium databases, virtual desktops,
development, build, and stage environments, code repositories, and product
prototypes.

For more information on T2 instance types, see

https://aws.amazon.com/ec2/instance-types/t2/.
 Get the Most, from the Best!!

 Sometimes unrequired resources are Utilization

kept running 120
100

 Leverage Amazon CloudWatch metrics 80

to find long-running idle instances

60
40

 Use Cost Explorer to find the costs 20

0
associated with entire projects or
initiatives

• Ease of spinning up resources sometimes means that unrequired resources are

kept running.
• Use Amazon CloudWatch metrics and CloudWatch alarms to find long-running idle
instances.
• Use tools such as Cost Explorer to find the costs associated with entire projects or
initiatives.
 Get the Most, from the Best!!

 Script to turn on and turn off selected AWS resources

 Best practice to reduce cost

stopinator stop [tags]

stopinator start [tags]

A stopinator is a generic term for any script or application written against the AWS
platform that looks for and stops unused instances. Such scripts are typically set up to
run during the evenings and on weekends. Employing a Stopinator can result in
significant cost savings to an organization, thus freeing up your cloud computing
budget for new projects. It's also a handy script to have around for one's own
personal AWS accounts, as it allows you to spin up resources you need at the
beginning of the workday, perform whatever experimentations you need to run, and
then shut it down when you are done at the end of the day.
 Get the Most, from the Best!!

Do we need an Amazon EC2 instance in order to run the stopinator?

 AWS Lambda
◦ Run code (e.g. Python) without provisioning servers
◦ Pay only for the compute time required
◦ Use this to write the stopinator script
 CloudWatch Events
◦ Define a source – either a recurring schedule or API events
◦ Define a target – something to be invoked (e.g., a Lambda function)
◦ Use this to trigger the script to run (e.g., every evening at 6pm)

For a complete example, see https://aws.amazon.com/premiumsupport/knowledge-

center/start-stop-lambda-cloudwatch/.

Here, let’s discuss how to stop and start Amazon EC2 instances at regular intervals
using AWS Lambda. For example, if you want to reduce your Amazon EC2 usage by
stopping and starting instances at predefined times or utilization thresholds, how do
you configure AWS Lambda and Amazon CloudWatch to do that automatically?

You can use a CloudWatch Event to trigger a Lambda function to start and stop your
Amazon EC2 instances at scheduled intervals.
Get the Most, from the Best!!
 Get the Most, from the Best!!

The AWS Billing Dashboard lets you view the status of your month-to-date AWS
expenditure, pinpoint the services that account for the majority of your
overall expenditure, and understand at a high level how your costs are trending. One
of the graphs located on the dashboard is the Spend Summary, which shows you how
much you spent last month, the estimated costs of your AWS usage for the month-to-
date, and a forecast for how much you are likely to spend this month. Another graph
is the Month-to-Date Spend by Service graph, which shows the top services that you
use most, and the proportion of your costs that that service contributed to.
 Get the Most, from the Best!!

Can export cost data to .csv file

If Consolidated Billing is enabled, you can use Cost Explorer to view costs across all linked
accounts, and monitor the individual daily and monthly spend for each linked account. You
can define and track budgets for your AWS costs, forecast your AWS costs for up to three
months out, and choose to receive email notification when actual costs exceed or are
forecast to exceed budget costs.

For more information on Consolidated billing, see

http://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/consolidated-billing.html.
 Get the Most, from the Best!!

AWS Budgets gives you the ability to set custom budgets that alert you when your
costs or usage exceed (or are forecasted to exceed) your budgeted amount. AWS
Budgets use the cost visualization provided by Cost Explorer to show you the status of
your budgets and to provide forecasts of your estimated costs. You can also use
Budgets to create notifications if you go over your budgeted amounts, or when your
estimated costs exceed your budgets. Budgets can be tracked at the monthly,
quarterly, or yearly level, and you can customize the start and end dates. Budget
alerts can be sent via email and/or Amazon Simple Notification Service topic.

Create custom budgets for costs Track monthly, quarterly or yearly spending
Refine to particular services, tags, linked accounts, etc. Receive alerts for actual or
forecasted costs, via email or SNS. E.g. “actual costs are greater than 50% of budget”,
“forecasted costs are greater than 125% of budget”

Budgets aren’t just for costs: they can also track service usage and RI utilization.
 Get the Most, from the Best!!

 Enabled in the AWS Management

Console
 Must be created in the us-east-1
region
 Include aggregated and service-
specific expenditures
Amazon CloudWatch
billing alarm ◦ For more sophisticated cost tracking by
tag, use Cost Allocation report or Cost
Explorer

You can monitor your estimated AWS charges using Amazon CloudWatch. When you
enable the monitoring of estimated charges for your AWS account, the estimated
charges are calculated and sent several times daily to CloudWatch as metric data.

Billing metric data is stored in the US East (N. Virginia) region and represents
worldwide charges. This data includes the estimated charges for every service in AWS
that you use, in addition to the estimated overall total of your AWS charges.

The alarm triggers when your account billing exceeds the threshold you specify. It
triggers only when actual billing exceeds the threshold. It does not use projections
based on your usage so far in the month.

If you create a billing alarm at a time when your charges have already exceeded the
threshold, the alarm goes to the ALARM state immediately.
Get the Most, from the Best!!
 Get the Most, from the Best!!

 Core Checks and Recommendations available to

all customers
 Additional Checks and Recommendations
available with Business or Enterprise support
plans
 Use Trusted Advisor Cost Optimization checks
to achieve a base level of cost savings
AWS Trusted  Sample Cost Optimization checks:
Advisor ◦ Idle resources (Amazon EC2 instances, RDS)
◦ Underutilized load balancers and volumes
◦ Unused Elastic IP addresses

AWS Trusted Advisory is an online resource to help you reduce cost, increase
performance, and improve security by optimizing your AWS environment, Trusted
Advisor provides real time guidance to help you provision your resources following
AWS best practices.

For more information, see

https://aws.amazon.com/premiumsupport/trustedadvisor/.
Get the Most, from the Best!!
 Get the Most, from the Best!!

Checks Amazon Elastic Block Store (Amazon EBS) volume configurations and warns when volumes appear to be underused.

4 of 7 EBS volumes appear to be underutilized. Monthly savings of up to $1530.20 are available by minimizing underused EBS
volumes.

Trusted Advisor scans your AWS environment and compares it to more than 50 AWS
best practices in five categories and provides recommended actions.
Recommendations include links to take direct action.

For information on Trusted Advisor best practices, see

https://aws.amazon.com/premiumsupport/trustedadvisor/best-practices/.
 Get the Most, from the Best!!

 Create and utilize tagging in AWS

 Identify cost reduction strategies/opportunities
 Configure your environment to monitor and reduce costs with
tools:
◦ Amazon CloudWatch
◦ AWS Trusted Advisor