If you’ve ever searched for a way to run a cron on AWS, surely you haven’t found anything to easily implement while being cost-effective. We’ve been on that road too many times as well.
That’s why we have developed CloudCron, a simple distributed, cost-effective, backwards-compatible, cloud-friendly cron to run on AWS. And, guess what, it is also now being open sourced!
A Cron is a time-based job scheduler that enables Unix users to execute commands or scripts automatically. These tasks run periodically at fixed times, dates or intervals. This automation is a useful resource but, until now, it was a little more difficult to do it in the cloud while having a maintainable setup.
The solutions you can find to run a cron on AWS may not meet your expectations:
- Cluster-based: some of the usual solutions required to build and maintain a complex, distributed, minimum of three nodes cluster. Some of these solutions let you schedule jobs, but their main purpose is to host more complex workflows than a simple cron.
- Lambda-based: since Lambdas can be scheduled, other options include creating a Lambda function.
- App server-based: some options require you to put the cron code in your app server and call it via HTTP.
- Running cron on an instance: while very simple, this commonly turns into capacity and deployment problems in the long run.
Those are no easy and cost-effective solutions to run a cron on AWS. So you may want to try our CloudCron solution.
The CloudCron solution: run a cron on AWS
Cloud is the new default. Thus, migrating your crons to AWS, one of the best performant public cloud providers, should be a piece of cake.
Our team of Developers and SysArchitects that design cloud-based solutions are usually challenged with the requirements to run a cron on AWS, and that’s how CloudCron was born. Now, we use it on a daily basis. We’ve designed CloudCron to be:
- Backwards-compatible. Just give it a cron file and it will run the same command you already have on the same server if you want. There is no need for reprogramming your job.
- Cost-Effective. You just need the minimal infrastructure: a worker node.
- Both scalable and parallelizable. Do you have more crons than a machine can handle? Just scale your worker nodes as needed.
- Ready and set for red/black deployments. Just deploy a new worker node with new code or patches. There won’t be any downtime.
- Not time-limited. If you were thinking about the Lambda-based cron solutions, you’d find out that they have a maximum running time of 5 minutes. CloudCron has no time limits for tasks execution.
- CD Pipeline friendly. You’ll be able to version control your own crontab and deploy it as part of your CD.
- Not just for Cloud. CloudCron is on-premise friendly. You’ll be able to also run your tasks on premises.
How does CloudCron work?
Yes, that sounded good. But you may want to know exactly how CloudCron works before using it. So moving on to the substance.
First, CloudCron has a “cron compiler” that transforms your cronfile into scheduled CloudWatch events that enqueue into an SQS queue. These events will be pushed into CloudFormation, so they can be managed as a whole. Thus, the CloudWatch events will inject a message for each occurrence of a cron event to an SQS queue that, at the same time, is getting polled by a worker process. This worker process runs on the same machine (or machines) where your old cron could execute its jobs.
Then, CloudCron executes the events on one or more instances. Easy, isn’t it?
If you just want to run your old crons on AWS, this may be the right solution for you. To install it, just read the installation guide that you’ll find in our Github and start running your CloudCron.
An Open Sourced CloudCron
As you have seen, we decided to share CloudCron with the OSS community. You can use it for free as many times as you need (and if you do so, please give us some feedback!). Also, feel free to contribute to the project as much as you want. Contributions are always welcome 🙂
If it is the first time you read us, you might not understand why CAPSiDE is sharing a tool such as CloudCron but we, at CAPSiDE, are Open Source at heart. Our company has been built on the back of open source software for more than 15 years. This is just part of the work we give back to the larger community. That’s why we try to participate in conferences and workshops promoting open source development. And that’s why we’ve contributed with a plethora of things we’ve created to Open Source (AWS and Microsoft Azure Perl SDKs, for example).
We keep on working on new Open Source tools that you’ll be seeing in the following months. So stay tuned, you might be surprised.