Development

Heroku vs. Amazon Web Services

20 Comments

There’s been a bit of discussion lately on deployment options. Much of the debate is centered around the relative merits of Heroku. We have some experience with Heroku and Amazon Web Services (AWS), so let’s dive into some comparisions.

Price

This is the only metric that counts for many clients, so it’s good to start here. The trouble with price comparisons is that no two services are exactly the same, so it’s hard to do an apples-to-apples comparison. So let’s see what we can get at the entry level.

Heroku

  • Free for the first dyno

You can’t beat free, and this is actually quite a good offering. One dyno is plenty to run many kinds of apps. Brochure sites, simple APIs, and blogs are a few of the many possible uses for this free dyno. But to make the comparison fair, we need to know what this free dyno includes and what strings are attached.

  • RAM: 512MB
  • Swap space: 1GB max
  • Storage space: 100MB max
  • Compute power: unknown, but feels like something between a micro and a small EC2 instance.

Drawbacks

  • Additional dynos/workers are $35 a month.
  • No other services can be run on dynos. Dynos are strictly for application processes. Databases, background workers, and other services usually cost extra through Heroku’s add-ons or third party services.
  • No way to increase RAM, storage, or CPU performance. Additional storage must be hosted separately through a service such as Amazon S3. App performance can only be improved by increasing the number of running dynos. Heroku automatically load balances and routes visitors to all available dynos.
  • No way to install system software. Heroku does provide some commonly-used packages such as Imagemagick, but if you need anything else, you’ll have to resort to hacks.

Amazon Web Services (AWS)

Amazon Elastic Compute Cloud (Amazon EC2) is the closest equivalent to Heroku’s dynos. One EC2 micro instance is approximately equivalent in terms of RAM and compute power to one of Heroku’s dynos/workers. However, in our experience, the performance of a full-stack Rails application on a single micro EC2 instance is not quite as good as on a single Heroku dyno. This could be because we were running database and workers on the same instance. We could have probably slimmed down the instance by removing unnecessary system processes, but instead we typically go with a small EC2 instance.
EC2 Micro and Small
As you can see, EC2 is much cheaper when paid for in advance. Let’s just go with a one year heavy utilization reserved small instance for this comparison.

  • $27.77 a month on average (after amortizing the deposit and paying for the usage over one year).
  • RAM: 1.7GB
  • Swap space: configurable (presumably up to the total amount of storage space minus root partition)
  • Storage space: 160GB
  • Compute power: 1 EC2 Compute Unit

Again, it’s hard to make a direct comparison, but some of these figures are considerably higher than Heroku’s (1600 times the storage space!).

Drawbacks

  • You have to deploy your application yourself, either through Chef recipes, Capistrano, or manually.
  • You have to administer the system yourself. EC2 has machine images of popular distros such as Ubuntu that are easily launched, but after that it’s up to you to keep it up to date and secure.
  • Scaling horizontally (i.e. launching multiple app instances) is not as easy as with Heroku where it’s just a matter of moving a slider on their web interface. You’d better get familiar with Chef if you want to scale up and down frequently. This seems like a big drawback but in practice we rarely adjust the number of running instances for an app.
  • AWS is more expensive for the basic offering. There is a free tier that will give you one free micro instance for the first year (only available when you first sign up) but this is not as generous as Heroku’s free, unlimited, single dyno apps.

Real life cost examples

Say we have an app that needs 10MB database storage, one worker, and SSL. With Heroku this will break down to the following:

  • $20 for increased database storage. Heroku’s free shared database only offers 5MB storage. They are rolling out a new option but it’s unclear what the specs are.
  • $20 for the SSL endpoint
  • $35 for the worker process
  • Total: $75 per month

To get the same thing on Amazon you’d pay the following:

  • $57.60 for an on-demand small EC2 instance (or $27.77 for a one year commitment; $17.69 for three years).

Admittedly it’s not a huge savings for the first month, but let’s say you need to add Redis and MongoDB. They both live mainly in memory, so it’s a good thing we went with the 1.7GB RAM on Amazon. We can easily run both of these services on our single small instance. On Heroku we’d need to add the following:

  • Redis To Go Small 100MB Instance — $25
  • MongoLab Small 0.50GB Storage — $10

With Amazon it’s easy from a price perspective to add or remove services. If you go with Heroku, you may have to ask your client for an additional monthly payment for each service you add, making the decision more difficult and time consuming.

Other considerations

Besides costs, you will want to consider whether or not it’s even possible to run your application on Heroku before going that route. If you need to run custom binaries or compile from source, you will have to figure out how to hack Heroku, or you may be out of luck. If you need to store temporary files you should know that Heroku’s ephemeral file system does not make that task easy. You could end up exerting a lot of effort only to eventually run into a brick wall. Personally, I’ve had to hack gems and try to find workarounds for many Heroku-specific issues.
On the other hand, if you know your application will fit within the limitations, deployment on Heroku is a breeze. The web interface is beautiful, the CLI client works well for the most part (except it can’t manage multiple Heroku accounts), and it’s easy to add other services through add-ons. But if you think your app’s needs might grow in ways that won’t be satisfied by simple horizontal scaling, you should definitely consider AWS. Of course, you can always start with Heroku and later migrate to AWS, but in that case you will have to configure your app for two environments and spend the time to migrate everything.

Other options?

In my experience, there are simply no other platforms that compare well with Heroku or AWS. There are some nice Heroku alternatives such as Dotcloud, but they suffer from coming after Heroku and thus having less integration with third-party products and less community support. Rackspace is the closest competitor to AWS, but it lacks the rich APIs and support that Amazon provides. Hopefully, we’ll see competitors catching up to Heroku and AWS and offering us more choices.

Conclusion

Both Heroku and AWS are excellent platforms. They are quite different in some key areas. Understanding what each offers is essential to picking the right platform for your application. There is no clear winner here. Personally I like to use both. But my (simplified) mental test for which one to use is this: small app–Heroku; large app–Amazon. The winning platform is whichever one most helps you achieve your goals through keeping your developers happy and productive while remaining affordable enough to be sustainable.

Editors note: Reed recently revisited the Heroku vs. AWS comparison. 

 

| | |

20 Responses

  1. gady says:

    What about EngineYard?

    • Reed Law says:

      I’ve used EngineYard as well, and it’s kind of like a managed EC2 with better support. It’s mainly geared towards Ruby on Rails applications. Fine choice if you need more support to deploy and maintain your app.

      • Ryan Porter says:

        Engine Yard has become a lot more than just Rails hosting. They bought Orchestra.io’s cloud PHP PaaS and integrated PHP support with the Engine Yard Cloud, and they also support Node.js.

        This article comparing Heroku and AWS is comparing apples to oranges, because AWS is an IaaS and Heroku is a PaaS. Engine Yard is also a PaaS, so comparing Heroku with Engine Yard makes a lot more sense. And where Engine Yard shines is support. If you need help with a Heroku app then good luck. But if you need help when you’re on Engine Yard then they can and will help you, especially if you pay for premium support. The premium support option seems expensive when you first see the price but it’s not. Compare that price to what it would cost to hire a staff of web operators who are available 24/7 and who have a network of experts to call on for help. Outsourcing your web operations to Engine Yard saves six figures per year at least by comparison.

        Engine Yard is a lower-level abstraction than Heroku, which is good sometimes and bad sometimes. It’s awesome that Heroku has a super high-level abstraction level so that you can think more about your app than about the machines. But every once in a while when you use Heroku, you really wish that you had SSH access to the machine so that you could just install your own packages or tweak your technology stack or whatever. Engine Yard gives you that.

        I would say that it’s for a more advanced user than Heroku, who wants the flexibility that Heroku doesn’t give you, and who needs the serious 24/7 support that you need for a truly mission-critical web service or web site.

        • Arvind says:

          I you read anything more than the title, you’ll see that it’s not a ‘head on’ comparison. It’s not like a direct comparison between apples and oranges. It’s more like apple flavoured ice-cream vs Orange flavoured ice-cream.

    • Colin Adams says:

      Hey Gady – We use both Heroku and Engine Yard for different applications. As both Reed and Ryan mentioned, the philosophies behind the two services are quite different.

      Reed does a really nice job of summarizing Heroku above. Depending on what you are doing, Heroku is very cool and very ‘cloudy’. I love that I can bring up instances (dynos) in mere seconds and then destroy them just as quickly. Very easy to then monitor load and respond. We use Heroku for running AirBop.com (an Android/GCM Push Messaging Server Platform) because of this. When we need to launch 100 servers, we can. When demand drops, we kill them off just as fast.

      However, it’s the drawbacks that Reed nicely summarized that can be a real problem for other use cases. That’s where I think Engine Yard shines. Rather than dealing with EC2 directly, you get a platform that eliminates that up-front complexity and yet you have full access to the servers. More importantly, you have a group of support people that are on your side when issues arise.

      We use Engine Yard for running Andromo App Maker for Android (http://www.andromo.com) because we need that total system control. Andromo is doing some rather funky stuff to generate Java/Android app source code on the fly, compiling against the Android SDK and then delivering a native Android app in minutes. If we had to worry about everything ourselves like we would with say just EC2, we’d spend a lot more time dealing with servers and less time building our product.

      And just to touch on what Ryan Porter said – the support options with Engine Yard are *way* better than what I’ve seen with Heroku. We were having some issues with database performance on Heroku, so I sent off a support request to shed some light on it. I basically got a link back to their database options webpage (which was absolutely useless from a technical standpoint) and had to spend a few days manually discovering capabilities on my own. Any contact I’ve had with Engine Yard support has been helpful and timely.

      In any case, good luck with your projects!

      Colin

    • Toby Osbourn says:

      I have been using EngineYard for a while now in my day job and am more than happy with them.

      As others have said they essentially provide a support level on top of EC2, but the support is fantastic and if you want to get your hands dirty you can.

      I have mainly used it for Rails hosting, but I will soon be deploying a Node application to it.

  2. another says:

    How about a comparison to Linode?

    • Reed Law says:

      I haven’t tried it yet. Whenever I get a chance to do so, maybe I can revisit this comparison.

    • Jon Takaki says:

      I just ran a benchmark of a small Linode against a micro instance on AWS. Since I’ve heard everywhere Linode was much faster, I was expecting my tests to indicate that.

      I set-up the same nodejs project on both under Ubuntu an then ran some benchmark with 1000 requests and 200 concurrent and another for just one request.

      The Nodejs projects uses Express and checks if a cookie is valid by doing an MD5 hash… This test showed that a micro instance was 30% faster for one request, and up to 3 times faster with “ab -n 1000 -c 200 …”
      Then for checking the validity for an HTTP POST login, the results were similar.

      So in conclusion an AWS micro instance can deliver much more requests than a small Linode for a Nodejs application.

      However Linode was much faster for installing software due to its, probably better IO performance.

    • Toby Osbourn says:

      I have had some negative experiences with Linode and downtime, their support isn’t brilliant either in my experience.

  3. Matt Bessey says:

    I’m surprised no one is mentioned Linode, we have just started using them and I’ve found for half the price of an EC2 small instance you get a lot more CPU power. Not as much RAM admittedly, but unlike Amazon, Linode lets you use whatever idle CPU power is available to you, meaning on the server I’m on at least I have successfully maxed out 4 cores for several minutes at a time.

  4. anders says:

    To me, the real value of AWS and services like it is the API that gives you the ability to bring up and shut down nodes programmatically. You can have your code check server load average, free memory, or task queue length and bring up additional workers automatically when a threshold is passed. No human intervention required. Or you can bring up additional workers via a nightly cron job to run daily number crunching tasks.

    If you’re not using that functionality and you’re just treating EC2 instances like regular, manually configured/managed virtual servers that you’d get from Linode, Slicehost, Prgrmr, etc, I don’t see any point in running on EC2. You’ll get better performance for less elsewhere. The draw of AWS is elasticity.

    • Reed Law says:

      The only way I’ve benefited from the API so far is having Chef automatically create EC2 instances. But this was mainly used for testing our deployment, not so much for scaling. I agree that this is a major part of the value of AWS, but I’m not so sure about the idea that better performance value can be had elsewhere. I recently tried out a dedicated server from Hetzner, and although very cheap, it has less RAM and what seems like slower response time than a comparable EC2 offering (although a large part of that is due to Hetzner being located in Germany). The Linode 1536 at $60 is not quite up to a small EC2 instance’s RAM and storage specs. On the other hand, both Hetzner and Linode offer a lot of free data transfer while EC2 only includes 1GB transfer out.

      • Chef is not super easy for folks that may not know ruby, yet I also use its recipes to provision images. But I am surprised no one has mentioned elastic beanstalk from AWS, I think that it actually fills a very big gap and makes AWS the de-facto #1 choice for me at least :)

    • They all offer the seeming same level of API control it seems except.
      1. I think Amazon is the only one that can do HA so it is a seamless increase in resources. But then again a good chef or capistrano script could setup another HA Proxy quite easily and still keep the cost at par with Amazon.

      2. Amazons services may have better integration with Rails, Capistrano, Chef etc than say Linode because of the userbase out there building Gems, Recipes and so forth. This one though I am not sure about.

      Late to the game but great post as we research this so I figured I would add my 2 bytes.

      Thanks
      Al

      • Reed Law says:

        There’s an issue with Amazon’s Elastic Load Balancers. It seems as if they are being run on micro-instances, although they are supposed to scale for burst traffic. It’s not clear how or when they scale so you’re better off getting your own instances if you want more control.

  5. Totally confused because i never heard about this both services.seeing your post and get some idea about this both services.

Leave a Reply