Every project Iâm working with starts with a setup of continuous integration pipeline. Iâm a big fan of cloud services, thatâs why I was always using Travis. A few of my clients questioned this choice recently, mostly because of the price. So I decided to make a brief analysis of the market.
I configured Rultor, an open source project, in every CI service I managed to find. All of them are free for open source projects. All of them are hosted and do not require any server installation Here they are, in order of my personal preference (first four are the best and highly recommended):
| Minimum price | Linux1 | Windows2 | MacOS3 | Pull requests4 | Log compress5 | Docker6 | Build | |
| Shippable☺ | free (!) | |||||||
| Travis | $69/mo |  | ||||||
| AppVeyor | $39/mo | ? | --- | |||||
| Wercker | $350/mo | |||||||
| SemaphoreApp | $29/mo | |||||||
| Snap-CI☺ | $30/mo | ? | ||||||
| Codeship☺ | $49/mo | ? | ||||||
| CircleCI☺ | $19/mo | |||||||
| SonoLabs | $15/mo | ? | ||||||
| DeployBot | $15/mo | ? | ? | |||||
| Vexor | ¢90/hr | ? | ? | |||||
| GreenHouseCI | $49/mo | ? | ? | |||||
If you know any other good continuous integration services, email me, Iâll review and add them to this list. BTW, here is a âfullâ list of continuous integration software and services.
By the way, a few platforms from this list contacted me and asked to review them again. Some even offered me money to put them higher in the list (kidding). Anyway, itâs up to you to decide whether itâs a good sign or not (I think itâs good, they care about their PR). I marked them in the list with âș emoji.
Best Four
Shippable was easy to configure since it understands .travis.yml out of the box. The user interface is easy to navigate since it doesnât have âsettingsâ page at all (or I didnât find it). Everything is configured via shippable.yml file in the repository. The service looks stable and robust, no complains so far. What is especially cool about them is that they allow you to build in a Docker container.
Travis is the best platform Iâve seen so far. Mostly because it is the most popular. Perfectly integrates with GitHub and has proper documentation. One important downside is the price of $129 per month. âWith this money you can get a dedicated EC2 instance and install Jenkins thereââsome of my clients say. I strongly disagree, since Jenkins will require a 24x7 administration, which costs way more than $129, but itâs always difficult to explain.
Wercker is a European product from Amsterdam, which is still in beta and thatâs why free for all projects. The platform looks very promising. It is still free for private repositories and is backed up by investments. They also have an interesting concept of build âboxes,â which can be pre-configured similar to Docker containers. It works rather stable for the last few months, no complains so far.
AppVeyor is the only one that runs Windows builds. Even though Iâm working mostly with Java and Ruby, which are expected to be platform independent, they very often appear to be exactly the opposite. When your build succeeds on Linux, there is almost no guarantee it will pass on Windows or Mac. Iâm planning to use AppVeyor in every project, in combination with some other CI service. Here is how I integrate Maven builds with AppVeyor.
Others
SemaphoreApp is easy to configure and work with. It makes an impression of a light-weight system, which I generally appreciate. As a downside, they donât have any Maven pre-installed have an old version of Maven, but this was solved easily with a short custom script that downloads and unpacks the latest Maven. Another downside is that they are not configurable through a file (like .travis.yml)âyou should do everything through a UI. They also support caching between builds.
Snap-CI is was a product of ThoughtWorks, an author of Go, an open source continuous integration server. It looks a bit more complicated than others, giving you an ability to define âstagesâ and combine them into pipelines. Iâm not sure yet how these mechanisms may help in small and medium size projects weâre mostly working with, but they look âcool.â There is also a very unfortunate limitation of 2Gb RAM per buildâsome of my Java projects fail because of that. Besides that, they donât give full access to the build server, for example we canât modify anything in /etcâit is a show-stopper for us.
Codeship works fine, but their web UI looks a bit out-dated. Besides that, they promise to work with pull requests, but I didnât manage to configure them. They simply donât notify our pull requests in GitHub, even though they build them. Maybe Iâll find a way, so far itâs not clear.
CircleCI I still donât know why my build fails there. Really difficult to configure and understand whatâs going on. Trying to figure it outâŠ
SolanoLabs looks rather immature and difficult to configure. They donât even support automatic GitHub hook configuration when new repository is added. However, their sales spams me rather aggressively.
Hosted-CI is for iOS/OSX only. They donât give anything for free, even for open source projects. I didnât have a chance to test them yet.
CloudBees is basically a hosted Jenkins. I donât really like Jenkins, thatâs why canât recommend this platform.
DeployBot doesnât even allow me to login via GitHub, huh? They seem to be more âdeploymentâ oriented, not just continuous integration.
Vexor looks nice and offers a rather unique billing modelâthey charge per build, not per month. I would definitely recommend to give it a try. I couldnât make it work thoughâŠ
GreenHouseCI is a CI platform for mobile apps (iOS, Android, etc.) Seems to be interesting, I just donât have a full scale mobile app to test it against.
gitlab-ci: will review soon
coverity.com: will review it soon and add to the list.
buddy.works: will review soon.
AWS Code Build and AWS Code Deploy: will review soon.
Ship.io is dead (as of 20-Sep-2016).
ZeroCI.com is dead (as of 28-Aug-2016).
Drone.io is not hosted any more, but is open source (as of 23-Jan-2017).
Hosted-ci doesnât look alive (as of 16-Apr-2017).
Magnum-CI.com doesnât look alive (as of 30-May-2017).
BTW, if you donât like the idea of keeping continuous integration in cloud, consider these on-premise software packages (in order or preference): Jenkins, TeamCity, Go, Strider, BuildBot.
Keep in mind that no matter how good and expensive your continuous integration service is, it wonât help you unless you make your master branch read-only.
1 This means that the platform can build your repo in Linux environment. Almost all of them do that by default, unless you configure them otherwise.
2 Some of them can build on Windows platform.
3 MacOS support means that an Objective-C/Swift product can be built there.
4 I mean GitHub pull request support here. Some of them can be integrated with GitHub and will build pull requests before they are merged. Build status will be visible in GitHub. A pretty convenient feature.
5 Logs compression is a critical feature, at least for me. Most of my logs are from Maven and without col -b they look too long and unreadable.
6 Looks like Docker containers must be supported by all of them, but unfortunately itâs not the case. Ideally, all builds should run in containers.
