Super basic approach on how to achieve cloud cost efficiency thanks to the agility in cloud environments.
Your very first days working with cloud environments can be a little confusing because you are not used to its different paradigm, and some of your early questions may have easy answers. Looking for advice, many companies ask us how to be cost-efficient when getting started on the cloud.
Even though this post may seem a little too basic, we think it may help those who have some early doubts, also helping some companies to save some money when they get started with the cloud. In “Cloud Cost efficiency 101” you will learn how to be Cloud cost-efficient in testing and preproduction environments on Amazon Web Services using simple AWS CLI commands.
Usually the main cost on infrastructure goes to compute capacity of servers. In AWS, when we talk about compute capacity we talk about EC2 instances, where the cost depends only of the actual hours you use each instance. Also, a bigger instance in terms of compute capacity is more expensive, logically. It is pretty common to use testing and preproduction servers only during working hours, so it does not make any sense to have these kind of environments up and running 24 hours a day. For that reason it’s interesting, in terms of cost-efficiency, to take advantage of how easy is to stop and start instances in the AWS cloud. For example, you are able to stop all instances during the night and start them back up in the morning. There are two different approaches:
- Long-lived instances (non-autoscaling)
- Instances inside Autoscaling groups (ASG)
Cloud Cost Efficiency: Long-lived instances
For long-lived instances, we can use a simple bash script whose execution can be scheduled in a cronjob. This cron would use AWS CLI commands in order to stop specific instances. Imagine that all your instances within an AWS account are tagged with the tag “Environment” with different values depending on the belonging environment: production, preproduction and testing. The script will simply check which instance-IDs have a specific tag (“Environment = preproduction” for example), and once we have their IDs, we will use the AWS API to send a stop signal to these EC2 instances. The script could be something like the following:
In the previous command we are filtering by environment tag, and once we have all the information of the different instances, we use JMESPath in order to obtain only the instance-ID. Once we have the instance ID, we send an EC2 stop-instance command to all these instances.
In order to start the instances, we had another script that executes the start-instances command:
Cloud Cost Efficiency: Autoscaling groups
For instances inside an ASG (AutoScaling Group) the approach will be different due to the fact that if you turn off an instance inside an ASG, the ASG will replace it with a new one.
ASG’s configuration have three different values that correspond to the number of instances within an ASG:
- Minimum: minimum number of instances within an ASG
- Maximum: maximum number of instances within an ASG
- Desired: real number of instances, which will be between minimum and maximum, and will depend on the autoscaling group policies and the use of the EC2 instances
Imagine that you have an autoscaling group with 4 instances (minimum=4, desired=4, maximum=8). In that case, instead of stopping instances, what you could do is to decrease the number of instances to 0 (minimum and desired = 0) at night, and restablish the normal values in the morning (increasing minimum and desired = 4).
In order to perform these actions, we are going to use again some available AWS CLI commands. In this case we will use what it is called “ASG scheduled actions“. These scheduled actions are based on universal cron expressions, in which we have to use UTC timezone. We will create two scheduled actions, the first one executed in the evening:
The previous command, executed only once, will create a scheduled action called “scale-down-night” that will be executed by AWS recurrently every day at 18:00 UTC (observe that the rule fires at 19:00 in Spain because of being on UTC+1 at that moment), and will decrease the number of instances of the ASG to 0. And this next one would be executed in the morning:
Take into account that once you have raised the number of instances within the ASG, the autoscaling policies will decide again the real needed number of instances (desired value) depending on the ASG load (CPU usage for example). Scaling policies based on CloudWatch alarms defines how the ASG scales out or in.
These are just very simple AWS CLI commands to start/stop instances easily, in order to save money on test/dev environments on AWS, and of course there are normally more elements to take care of in a full preproduction environment. In CAPSiDE we offer to our customers our own powerful tools in order to be able to automate the start/stop of different environments (including long-lived instances, autoscaling groups, RDSs…)
However, you can create powerful scripts with more complex logic and intelligence using your favorite programming language. AWS provides different SDK’s for almost all programming languages: Python, Ruby, JAVA, PHP, .Net, Go, C++, Node.js… By the way, if your favorite programming language is Perl, take into account that our CTO at CAPSiDE Jose Luis Martinez (aka Pplu) has created the Perl SDK for AWS (called PAWS). You can find more information at CPAN and the actual code on Github – AWS SDK Perl.