Migrating from IronWorkers to AWS ElasticBeanstalk

For running applications based on messages, we need an efficient and reliable background infrastructure. Iron.io’s IronWorker fulfilled those needs for quite a long time with affordable cost. As time progressed, our needs for higher reliability increased but the costs for a private iron cluster were prohibitive. When we started to have a feeling that IronWorker will not fit our needs anymore,that’s when we started our research on the self-hosted environment so that we can tweak the performance on our own while keeping costs low. IronWorker failed for the following things, that gave us intensified feelings towards switch

  • Technical Issues
    • ECONNRESET – unpredictable connection issues due to an overloaded public cluster
    • Issues with IronCache – unpredictable downtime and service devolvement
  •  Support – It is my experience that any software cannot be perfect in a one shot but good support keeps the audience engaged. This is where I think that Iron.io stopped showing interest due to the cost of troubleshooting within a public cluster to them.
  • Pricing – IronWorker has increased their pricing that we did not even think of. IronWorkers are the most feature complete technology in this space but compared to the resources they provide with Shared Instance plan, running workers on self-hosted infrastructure will often be more affordable if you are willing to give up on some of their features.
  • Loss of Freedom – Even if we purchase the dedicated IronWorkers plan from Iron.io, we do not get the access to the server. So even if we want to check for the things in some urgent situation instead of waiting for the support, we will not be able to do it.
  • Resources – Pricing and resources are complementary points. As mentioned above in Pricing section, the price that Iron.io is charging for their features could be difficult to pay for some.

As an immediate alternative to IronWorker, we thought of using AWS Lambda but it failed as it has a limitation on the total time for which a single job can run. AWS Lambda allows a single job to run for 5 mins maximum and our job needs minimum 30 mins (network calls mean that it is not in our hands to decrease time) to finish successfully. So our search continued and we struck to AWS Elastic Beanstalk (EB).

EB can be configured as Web Tier or Worker Tier. Between the two, Worker Tier fits our needs and is quite flexible so that any application can be ported to it. Of course, our worker code did not work as is, we had to make changes. We were able to keep 80% of our code exactly the same and only change 20% of it.

Things we liked about AWS EB

  • Customizable – AWS EB provides full control over the infrastructure so that we can configure the things according to application needs. Following are the few things you need to configure
    • Visibility timeout
    • Maximum number of retries
    • Inactivity timeout
    • Error Visibility timeout
  •  Robust and Fast – Once you tweak above parameters according to your application needs you won’t need to do many things and leave the other things to EB. Being the dedicated environment and due to features like AutoScaling, you don’t need to worry about the resources.
  • Logging – EB is fully customizable so you can stream your logs to the AWS Cloudwatch logs, another service from AWS or any third party service like logentries or papertrail or your ELK setup.

Things that AWS EB falls behind

As I mentioned earlier no software is perfect. There are few complaints regarding AWS EB setup but they are manageable at the end.

  • Deployment time – The time required to deploy the latest code on AWS is much greater than IronWorkers i.e almost 5-10 minutes. Although we have managed to reduce this time by deploying our worker via docker images instead of zip files.
  • Autoscaling Time – At the time of scaling needs, it takes a long time to boot up the new instance and start processing the jobs. It takes almost 8-10 minutes for it. We haven’t found a way to improve it and clearly Iron.io excelled in this regard with as little as a second to provision containers.

Things that we will miss from Iron Workers

  • Scheduler – IronWorker has a good scheduler for scheduling the jobs. This feature is available in AWS EB via cron.yaml in .ebextensions directory. But it is easier to manage scheduling in Iron.io than AWS EB.
  • Isolated Environment – IronWorker separates every worker via a docker container. The same VM runs each various workers based on payloads in our AWS EB so the memory is not isolated across runs as it is in IronWorkers. But yes, it is in our roadmap to improve.
  • Nice Dashboard – We can track the job status via IronWorker’s nice dashboard. In AWS EB infra, we have managed to get status with our custom code that pushes status to ElasticSearch (ES).
  • Pause/Resume Job Processing – This is a crucial feature from Ironworkers which we will miss. In AWS EB the messages are fetched by AWS SQS agent for processing them, and managing the queuing service for pause/unpause is not an option anywhere on the AWS dashboard.

Conclusion

AWS EB can be a real cost saving option for those who are ready to miss out on a few features that IronWorkers provide.

Things Todo in our RoadMap

  • Get Real Time Job Status
  • Pause/Resume Job Processing

Leave a Reply

%d bloggers like this: