There are a lot of CI/CD tools on the market today. Few have the rich history and community support of Jenkins. However, many cloud vendors have come out with their own competing solutions for enabling CI/CD in cloud-native projects. Today, I look at how AWS CodeBuild stacks up against Jenkins - and discuss which one you should consider using for new projects.
As I've discussed before, continuous integration and continuous delivery (CI/CD) are processes that increase software project build quality and velocity. CI processes focus on validating that new code builds successfully and clears automated functionality and quality checks. CD ensures a new set of bits can be staged for deployment through the various stages of an application's build pipeline.
Jenkins is a continuous integration and continuous delivery (CI/CD) framework. It's one of the oldest CI/CD tools: Sun Microsystems developer Kohsuke Kawaguchi developed the first version back in 2004. Today, the company CloudBees owns and administrates Jenkins as an open source project.
Jenkins takes a bare-bones approach to CI/CD. Its main role is orchestrating the execution of common test frameworks and custom build/deployment scripts.
But within that simplicity lies power. With Jenkins, you can create build and deployment pipelines that match the complexity of your project. If an open source tool can be called from a Linux or Windows command-line, it can be integrated into a Jenkins project.
Jenkins can scale to handle large workloads. Jobs can be allocated by Jenkins controllers to nodes, allowing multiple build tasks to run in parallel across multiple machines.
Jenkins also sports a plugin architecture that enables third-party developers to extend the product's core functionality. Thanks to its popularity and longevity, there exists a rich ecosystem of plugins that teams can bring into their own projects.
The original Jenkins offers a very minimalistic UI to users. However, in recent years, CloudBees has invested much time and resources into a new user experience called Blue Ocean.
Some may argue that Jenkins is more difficult to set up and administer than other CI/CD solutions, such as CircleCI or GitHub Actions. And that's true, as far as it goes. But there are solid reasons to choose Jenkins over other CI/CD software.
First, Jenkins' history and ubiquity makes it a proven, reliable tool for CI/CD. That's made it the default CI/CD framework of choice for many developers. A 2018 report by Datanyze estimated that 68% of the world's continuous integration builds ran on Jenkins. The massive community surrounding Jenkins makes it easier to find help when you encounter blocking issues.
Jenkins is also non-opinionated, which makes it flexible and versatile. You can construct any type of pipeline you need using any mix of commands, frameworks, and programming languages that you desire. Its open-ended architecture means you can use it for a large variety of workloads. When I worked at AWS, for example, my team used it to engineer a Git-based publishing system for training content, complete with pushes to our our third-party textbook provider and handoffs to our localization team.
Finally, Jenkins is cheap! While CloudBees offers various enterprise CI/CD services, any team can pick up and use Jenkins at zero cost. This can be attractive for companies with intensive CI/CD needs that find their budgets for commercial tools increasing rapidly with usage.
If your team has the time to invest, or needs the flexibility that it offers, Jenkins is a solid choice for your CI/CD needs.
While Jenkins is a solid choice, it's by no means the only one. If you're an AWS shop, there's also AWS CodeBuild, Amazon's solution for enabling CI/CD.
AWS's complete CI/CD solution is part of a suite of tools, with CodeBuild at its center. First, you define where your code is sourced - either AWS CodeCommit (AWS-hosted Git), GitHub, or BitBucket. Developers define a BuildSpec file (build.yml) that dictates the steps in the build process, and where to store the resulting output (the build artifacts).
Once a build succeeds, developers can choose to deploy it with one or two tools. They can configure AWS CodeDeploy to deploy their application to a fleet of Amazon EC2 instances. Alternatively, they can construct a multi-step test and release process across multiple stages of development with AWS CodePipeline. Development teams can also create artifact repositories for handing off their built packages to other teams or external deployment tools.
Jenkins vs. CodeBuild: The Breakdown
So, which one wins? Let's compare each of the tools across four different vectors: features, installation & management, ease of use, and pricing.
Jenkins is popular for a reason: it can handle practically any workload. Jenkins has kept its design simple in the name of enabling flexibility. Jenkins jobs are basically a set of command-line calls that shell out to whatever utilities and applications are available on the agent machines.
Additionally, Jenkins has a powerful feature, Pipelines, that enable defining multi-step build and release processes through a Groovy-based Pipeline DSL. Using Pipelines, you can create reusable deployment pipelines that are shared across multiple projects. Many large companies have used these features of Jenkins to speed up application development and deployment. At Expedia, for example, Jenkins powers the release of over 4,000 apps.
Finally, since Jenkins has been around for ages, it has a rich ecosystem of plugins that extend its functionality. If you're thinking of implementing some complex custom task in Jenkins, odds are someone's already beaten you to it.
How does AWS CodeBuild compare? Fairly well, actually. CodeBuild and its associated brethren hit all of the major points you'd expect from a CI/CD system. But there are two major points to consider.
First, while AWS' CI/CD solution hits all the high points, it's not necessarily a best-in-class contender in any single area. Many third-party vendors implement parts of what CodeBuild and associated tools implement with a better UI and more features (e.g., CircleCI for continuous integration, or Artifactory for artifact management).
Second, customization is a bit harder to manage. With Jenkins, you can implement customizations via command line calls. (Or you can build a plugin if you want to get really fancy.) With CodeBuild, customizing your build processes means spinning up your own Docker build container. It's a bit more involved and a bit cumbersome to debug should things go awry.
Finally, if you're a multi-cloud shop, the thought of anchoring your CI/CD capabilities to a single cloud provider's platform is likely enough to give you pause. Jenkins lets you remain cloud- and service-agnostic.
Installation & Management
When it comes to installation and management, AWS CodeBuild wins hands-down. Since it's Software as a Service (SaaS), there's zero installation or patching required. Onboarding to CodeBuild requires onboarding your project - not standing up additional CI/CD infrastructure.
Jenkins is the opposite. At a minimum, to get started with Jenkins, you need to install and maintain a controller. For best performance, you'll likely want to set up two or more build agents to do the actual heavy lifting. And if you need Jenkins to be highly available and always running? That's a whole other set of work altogether.
Installing Jenkins doesn't just mean keeping a farm of controllers and agents up - it also means regular patching. Remember that rich ecosystem of Jenkins plugins I mentioned earlier? It comes with a downside: security vulnerabilities. Since plugins are community maintained, many plugins with severe security issues end up never getting patched. In 2019, one security researcher alone found vulnerabilities in over 100 plugins.
Don't want to manage Jenkins yourself? CloudBees, the key company sponsoring Jenkins, has developed its own CloudBees Platform that delivers "Jenkins at enterprise scale". However, you'll have to pay for it: basic pricing starts at USD $300/month for a team of 2-5 users.
Bottom line: If you're doing serious work with Jenkins, you need someone to setup and administer it. This means either hiring someone to do it or paying a company like CloudBees to do it for you. By contrast, CodeBuild has a much lower barrier to entry and doesn't require dedicated admin expertise to keep it healthy.
Ease of Use
Earlier, I described Jenkins as "flexible". To be more blunt, its historic interface is...a little bit clunky. The first time I saw it I thought, "This is definitely a UI created by Linux developers".
As such, it can take a while for a Jenkins newbie to learn their way around. Truthfully, though, it's not that bad once you get used to it. And the Blue Ocean interface goes a long way toward simplifying the user experience.
Similarly, the AWS CodeBuild interface is decent. My only complaint is that, since the pipeline story is spread across multiple AWS services (CodeCommit, CodeBuild, CodeArtifact, CodePipeline, etc.), the experience is inconsistent.
For those who want to script their DevOps pipelines, both Jenkins and AWS CodeBuild have command-line tools that make this possible.
Jenkins has a great pricing structure: free! Of course, you'll need to pay for whatever underlying hardware or cloud services you use to host it, such as virtual machines, Docker container clusters, networking, and storage.
By AWS's own estimate in its Jenkins build guide, it'll cost you about $91.84 a month to run a single Jenkins build server (combined controller/agent with no additional build agent machines).
However, this assumes you're running an m4.large Linux server continuously throughout the month. If you only ran it during business hours, you could cut your utilization - and your cost - down to 20% of that, or around $18.40/month. If you need a dedicated, 24/7 build server, you can still shave the cost down by about half if you buy reserved instances.
With AWS CodeBuild and related tools, you pay according to AWS's utility billing model, paying only for whatever capacity you use. And in terms of AWS services, CodeBuild is relatively cheap. If you ran 200 builds a month, with each build lasting around 5 minutes, you'd only spend about $15/month for your build server.
Like I've already mentioned, most teams will use CodeBuild in conjunction with other services - e.g., Amazon S3 for storing build artifacts, CodePipeline for creating end-to-end deployments across multiple stages of development. This incurs extra costs.
However, these costs are pretty minimal. For example, a pipeline in CodePipeline only costs $1/month. And services like CodeDeploy only charge for the EC2 instance capacity you use - money you'd have to pay for hosting anyway even if you were using an alternative like Jenkins.
Overall, while you can restrain costs with Jenkins, AWS is easily the more economical option - especially when you factor in the ease of administration.
Jenkins vs. CodeBuild: Weighing Your Options
So which solution do you use? As usual, it depends on your application's needs and/or tolerance for undifferentiated heavy lifting.
Personally, I still see Jenkins as a solid choice under any of these circumstances:
- Your projects are complex in nature and you require Jenkins' Swiss Army Knife flexibility
- You need to manage a multi-cloud/hybrid strategy
- Your team knows Jenkins and can be productive with it right out of the gate
- You're concerned about cloud vendor lock-in
Additionally, your team should have the sysadmin know-how to host Jenkins yourself or it's economical for you to license CloudBees Platform or a similar Hosted Jenkins solution.
For other projects that are AWS-only, CodeBuild in conjunction with CodePipeline can handle complex projects without the overhead that comes with standing up Jenkins and keeping it secure.
An Even Easier Approach to Deploying Your Apps
Both Jenkins and AWS CloudBuild assume you have a DevOps engineer on hand who's expert at setting up CI/CD pipelines. But as we discussed previously, it's tough times for companies looking to hire DevOps talent.
We created TinyStacks to address this gap. TinyStacks goes further than solutions like Jenkins or CodeBuild by automating your entire DevOps strategy. We stand up all of the infrastructure your application requires to run on AWS automatically - no previous AWS or DevOps experience required. Contact us today to see how we can automate your deployments!