The growth of Travis CI has been tremendous over the last year. We’ve seen lots
of projects adopt it as their continuous integration platform, and we couldn’t
be more thrilled about that.
Along the way, we discovered pain points in the Travis architecture and in the
way the platform handles fairness in builds. Big projects with big build
matrices tend to cause our queues to clog up and make other projects wait until
they’re fully done.
With bigger projects tending to have longer builds, that caused a bit of worry
with lots of smaller projects that had to wait in the queue for the bigger
projects to build.
We’ve been pondering for quite a while how we could solve this problem and
introduce more fairness into the system, allowing smaller projects to build
quickly and giving all projects a fair share of build resources on Travis CI.
We’re introducing a limit on the number of concurrent builds that every project
can run on Travis CI. So far, we’ve immediately queued a project’s entire build
matrix onto our queue, ready to be picked up by a worker. That’s one of the
basic reasons why big projects with big matrices tend to make smaller projects
wait. It’s a first-in-first-out queue, so they’ll patiently wait until all of
the builds are done and it’s their turn.
With the new logic in place, one project can run five concurrent builds. As soon
as one is done we’ll automatically push the next. If we have capacity and a
project still has five builds running we’ll push another project on the worker
queue. That way, we ensure that a certain level of fairness allows all projects
to build as quickly as possible while making sure that a small number of
projects can’t fill up the queues for everyone.
We’re aware that this might cause builds for projects with bigger build matrices
to take a bit longer, but we think the overall fairness introduced by this
change makes Travis a place that’s evenly shared by the community.
We’ll be watching how that change works out in production, and fine tune the
number of concurrent builds if necessary. The number five is something we chose
based on the overall fairness and the build capacity we have available but it’s
not written in stone. We’ve also been thinking about taking into account the
general runtime of builds for a particular project and be smarter about
scheduling by giving projects with faster test ways a head start. But that’s
just off the top of our heads for now, we’ll keep an eye on how things are going
with the current change in place.
Technicalities
Now that we’ve looked at the fairness side of things, let’s look at the
technical side of things.
So far, we’ve pushed builds immediately onto our RabbitMQ queue as they come in.
This had the advantage that they’d be popped off in the order they arrived in
Travis, and we wouldn’t have to worry about scheduling.
But it also meant that our queues would have to be kept in sync with the
database. Should we lose the jobs queued in RabbitMQ for some reason, we had to
manually queue them. With the new logic, the database is the single source of
truth and jobs are only queued if there’s capacity available to build them.
Instead of pushing a job immediately on the queue, it’s now only stored in the
database. A job is now scheduled based on the heartbeats we receive from a
worker. If a worker signals that he has build capacity, and we find a build
that’s ready for the worker’s language queue, it gets pushed and built. Within
that logic we embedded the limit on concurrent builds along the way. If a
project has reached its limit, another one that’s available gets pushed instead.
All this has the benefit that we’re starting to move into a direction where
Travis is agnostic to the message transport used between the system’s
components. We could use ZeroMQ or simple HTTP instead, because we’re doing
simple inter-process communication instead of relying on the ordered nature of
AMQP, because exact order is not important anymore.
Another upshot of this is that we don’t need to do manual requeing anymore,
the new logic will automatically pick up any build that’s ready to be built.
We have several areas in Travis where order is still important currently, but we
already have ideas on how to tackle them. We’ll keep you posted on the technical
details!
On a fun side note, the changes described here were deployed earlier today.
Here’s a graph of the average run time of handling worker updates went through
the roof, middle finger style, until we added a database index:
Today’s motto:
The Bottom Line
Introducing a quality of service level to Travis CI doesn’t only introduce a
predictable level of fairness into the community platform. It’s also the
groundwork for Travis Pro, where we put a limit on and will charge based on the
number of concurrent builds for an organization.
As development on Travis Pro, our continuous integration platform for private
repositories, hums along we thought we’d give you an update on how things are
going and what kind of new features make their way into it.
Last week we rolled out a new feature that’s pretty much invisible to the user,
but still important to make Travis Pro more user-friendly: automatic generation
of deploy keys for private GitHub repositories.
Why is this important? Private GitHub repositories can only be accessed using an
SSH key (there’s a secret other way to access them, but we don’t dare talk about
it in public). If we only rely on a user’s SSH key, that would compromise
security by allowing access to all his repositories, so we need to add an SSH
key that’s solely for Travis to use.
Until last week, our fall-back was to use a little script that generates a
new deploy key for you and generates a .travis.yml file that includes the key
and instructions on how to manually add it to your GitHub repository.
We were not happy with the fact that it’s more work for people to start using
Travis Pro and that it could potentially compromise security should a customer
decide to give an external party only access to a snapshot of the source code
but not to the repository. Accidentally handing over the deploy key might grant
unwanted access to the repository.
So we added a neat feature that automatically generates a deploy key for you and
adds it to the repository, and only this particular repository. You’ll get an
email from GitHub notifying you of the fact. When we receive a push from a
repository that doesn’t have an SSH key in the .travis.yml or in our system yet,
we generate one and update the repository configuration on GitHub with the new
key.
That way we don’t break existing projects that still have a .travis.yml
configured and still allow for you to specify a key in the .travis.yml if you
need to.
If you’re already using Travis Pro and have your deploy key set up in the
.travis.yml, there’s an easy way to migrate. Just delete the source_key from
the .travis.yml and remove it from the GitHub deploy keys as well. We’ll
automatically generate a new SSH key for your repository the next time you push
a commit.
Curious about Travis Pro? You can still donate to our Love
campaign to get on the fast track to using it before
we launch it to the public! If you donated, just hit us up on
support@travis-ci.org, and we’ll get you hooked up!
Travis CI was designed to be an awesome testing service for anyone and everyone. With over 16,000 projects
on Travis to date, it is not uncommon to see people pushing the boundaries and building projects Travis CI does not have native support for.
Over time we noticed that some languages gain enough traction on Travis that it makes sense to provide some sugar so that configuration and setup becomes as simple as counting to three. From fairly early on
we’ve seen C and C++ projects being built and tested on Travis, as well as a surge of Go projects as of late.
Today we are happy to announce first class support for C, C++ and Go projects.
C and C++ support
To tell Travis CI to use the C builder for your project, specify the following:
Historically, Travis CI environment only had GCC (currently 4.6) preinstalled but with the first class support for C and C++ project, we’ve added Clang (3.1) and introduced a way to switch compilers using the compiler key in .travis.yml.
For example, to build with Clang you just need to add the following to your .travis.yml:
compiler: clang
or both GCC and Clang:
compiler:
- clang
- gcc
Testing against two compilers will create two rows in your build matrix, or possibly more depending on your other configuration settings. If you are familiar with how testing against multiple Ruby versions or JDKs works on Travis this should sound familiar.
During each test the Travis CI C builder will export the CXX env variable to point to either g++ or clang++ and CC to either gcc or clang.
Although the Go community is young and practices around continuous integration for Go libraries and applications are still being discussed, we have had many people shout out for first class support in Travis.
Travis VMs are 32 bit and currently provide
go 1.0
core GNU build toolchain (autotools, make), cmake, scons
Go projects on Travis assume you use Make or the Go build tool by default. If there is a Makefile in your repository root Travis will just run make, otherwise it will run
go get -d -v && go build -v
to build your project’s dependencies and
go test -v
to run tests.
To tell Travis CI to pick the Go builder for your project, specify the following:
Our documentation guides cover Go support on Travis CI. There is no support for multiple versions of Go because at the moment as the only released version of Go is 1.0.
