When I started collecting some facts for this year’s retrospective, I couldn’t
believe how many fanstastic, game-changing and also humbling things have
happened in just a single year.
If Travis CI has been a toddler when Josh and I
started touring conferences in 2011 to tell everyone about the idea and vision
then in 2012 it was a kid growing up, going through some heavy duty rock’n’roll
and all night party times.
In 2011 the initial experiments
formed into a vision: We wanted Travis CI to be for tests and builds what
RubyGems is for distribution. But not just for Ruby, for any language. We
wanted to target Open Source code first. And we wanted dead-easy and public
continuous integration to become a standard for Open Source.
Over time we got more and more serious, also because we got such great and
encouraging feedback. Since we were quite limited on resources, we’ve had to
adhere to a “build the simplest thing possible” rule pretty strictly. The
result was a rather simple app that showed an exciting vision, but also already
proved useful enough for people to rely on on a daily basis.
Then in 2012 Travis CI saw rapid growth.
It became obvious that we’d need a much better foundation, so we could scale
things out with the exploding demand. We’ve collected some money from the
crowdfunding campaign. This allowed us to grow
our team and implement the things we’ve promised to build.
What we’ve been working on in 2012
Here’s what we’ve been doing in 2012 in terms of code:
If you’re really interested you can play around
and add/remove team members, repositories or change the scale. I’ve built this little
app to get a good overview for this retrospective.
You can see nicely how the crowdfunding campaign (orange) took quite a few
resources in the first weeks of the year, but then zeroed out. Instead we
started working on Travis CI Pro (blue) which took a huge part, and still does.
In addition, somewhere around June we started working on the new Ember.js web
client as well as the new API (yellow, see below) and split up the app more and
more (red) towards the end of the year.
And if you really want to stalk us more then you can find even more detailed information
in a feature list and
a summary of our commit history.
Haha, don’t worry. I’m going to sum it all up for you. :)
The Travis Crowdfunding “Love” Campaign
Conceiving, planning, designing, realizing and launching the crowdfunding
campaign including the Stripe payment integration,
founding a company (so we could legally accept money), production of
ringtones and a few rounds of private feedback from fellow developers took
about two months in total, one developer mostly.
We were already amazed by the fantastic amount of support and great feedback
that we were getting while we were putting the campaign together. But when the
site went live in February and raised a remarkable amount of money in no time,
we were just blown away and plain humbled.
If you have a spare minute then please take the time to review our fantastic
sponsors and the amazing list of private donors.
The overall revenue of the crowdfunding campaign to date amounted to roughly
125,000 USD. Without this money Travis CI would not be what it is today.
There’s so much to say about the crowdfunding campaign that doesn’t fit into
this post. We’ll write it up. Expect a more detailled summary early next year.
The Travis Team
In the course of 2012 the team behind Travis CI has more than doubled.
If you go to Berlin and ask around who people would really like to work with -
who do you think would be on the list? I’m obviously biased, but I’m sure
Konstantin and Mathias would rank very high within the Ruby community.
To say the very least, we were super happy that with
Konstantin and Mathias
two of the best Ruby developers we know joined the team in January full time.
It took us a while to change the company contract and fully get them on board,
but finally in April we were all set. They’ve changed the face of Travis CI
entirely. Having these guys join the team was the best thing that could happen
to the project.
Then in March and April one of our earliest and most supportive sponsors,
Enterprise Rails, sponsored Lucas to work
on Travis CI on a halftime position, fixing bugs and adding features.
Lucas is a Rails developer at Avarteq (and an
active coach in the Railsgirls Berlin group,
high five!) and helped us a lot, fixing bugs and adding
features.
In October Engine Yard started sponsoring a
fulltime position for Piotr in the most broadminded
way,
as part of their Open Source Grant program. Piotr
is a fantastic Ruby developer. He has been active as a Rails core member and
worked on distributed applications, as well as Ember.js based clients in the
past, so he was the perfect addition to our team. We are super happy to have
him on board, and super thankful for Engine Yard’s sponsorship!
Michael, one of the first members on the
team, luckily was able to continue to maintain our VM cookbooks, one of the
most important and mature components of the system, in the most dilligent and
trusty way.
We’d also like to mention Henrik, who’s done some tremendous work on community
support, tickets and a lot of bug fixing over the last months. He’s also built
the Mac OS X toolbar “watcher” app for Travis CI
as well as Travis Lite, an alternative, light-weight
web client for the Travis API.
More languages, pull request support, secure env vars and build artefacts
One year ago, Travis CI offered native support for six languages:
Clojure, Erlang, Node.js, PHP, Ruby and Scala. Today we are proud to say that
(after adding C/C++, Go, Groovy, Haskell, Java, Perl and Python) there is
native support for 14 different languages
on Travis CI, including support for JVM switching and a generic JVM language
builder. Besides these officialy supported languages, there are also projects
testing EMACS Lisp, Smalltalk, Bash and many other languages
on Travis CI.
We believe this is huge.
Next to that, the most exciting and popular new feature on Travis CI is
the pull request support
and later the tight integration with GitHub’s UI,
that has been added. Konstantin has done an amazing job at this, also adding a
new layered GitHub API client that now makes our
life much easier at Travis CI.
Then later this year, Piotr added two more features that gave tremendous
value to Travis CI, as they open up an entire new space of usecases and
possibilities: secure env vars
(so users are now able to add sensitive data to their public .travis.yml config
files) and (building on that) build artefacts.
Pure JS client and a shiny new API v2
If something is broken and then gets fixed we tend to entirely forget about
that unfortunate previous situation because we’re so happy with the new one. However,
we still remember how many issues we had with both the old API and the old JS
browser client.
We’ve started working on a new Travis CI web client in early June and a new Sinatra
based API app shortly after that. Since we also had to tackle so many other
things in parallel, the whole thing wasn’t ready to launch into beta
until October.
But we were super happy with the result, as it’s all much faster, much
easier to maintain and add features to it. We’ve since ported the client to
Travis CI pro
and fixed a good number of edge cases around the new JS based OAuth sign-in
process.
Travis Hub and the army of apps
One year ago Travis CI consisted of a web app, a bunch of worker machines and a
central message crunching app called Hub. Today there’s an entire small army of
apps that all share their their part in getting your builds run:
- Listener picks up requests from
GitHub (created Dec 2011).
- Gatekeeper configures them and takes care of syncing user data with GitHub (Oct 2012).
- Enqueue queues build jobs based on a rate limiting strategy (Oct 2012).
- Hub still is a central node that picks up state
changes and triggers events.
- Tasks sends out notifications to external targets
(like Email, IRC, Campfire etc., created Aug 2012).
- Logs does nothing else but processing logs that
are streamed over from the workers (Aug 2012).
There are also other apps for serving the static web client and
serving the HTTP API.
Moreover, we created other tools for things like
administration,
single sign on,
worker maintenance and so on …
Breaking up Travis CI into lots of tiny apps that communicate via AMQP, Sidekiq
and HTTP made it much easier to scale things out, spot issues related to a
certain aspect but also work in parallel without accidentally stepping on each
other’s work.
Automation and Visibility
Even with the rather simple setup we had a year ago, when things went wrong it
often turned out to be quite hard to tell what’s actually going on in the
system. That situation would not get any better once we’d split things up
into separate components even more.
Being the devops hero he is Mathias immediately jumped at tackling that
situation. He changed and cleaned up the way our apps write log output,
centralized them in Papertrail, added metrics
everywhere, piped them through the logs to Librato so
we’d get some pretty graphs and assembled them on a nice dashboard.
Over the next months the situation improved step by step and we’re now able
to tell what’s happening
and isolate issues
rather quickly even though our app now consists of many more components.
Travis Pro
And finally, we’ve worked a lot on turning this platform into a paid
service so we could eventually allow everyone who had been asking for this to
give us their money … so we could continue providing this service for Open
Source for free and also make it even better, more useful and exciting.
Obviously we can’t talk much about the exactly way this was realized, but we
are able to re-use quite a bit of the code that is also powering the OSS
version of the platform. The two services are entirely separated and share no
resources other than code though.
Figuring out all these bits and pieces took quite a bit of work. Mathias has
been working (next to lots of other things, but still) 2.5 months on
integrating payments, coupons, billing etc. alone. A bunch of things need to
work in a slightly different way for the private service, so lots of
adjustments had to be made.
We’re still in a closed, private beta and we have a number of big improvements
in the pipeline before we’ll be able to entirely open the service. But we can
say proudly that our users are already pretty happy with it, given the
feedback
we have.
What’s up for next year
In 2013 we will mostly continue the path that we’ve taken in 2012 and improve
the existing service. Reliability, flexibility and robustness are on the top of
our list. Further improvements to usability and performance, adding missing
features and further build the community around Travis CI are other goals on
that list.
But we also want to look into fields that we haven’t quite touched much, yet,
like simplifying support for continuous deployment and other automation steps
or adding an API for third party apps. We have some great ideas in this space,
so be excited :)
Obviously our main priority has to be put on shipping Travis CI pro, so
we’ll be able to pay our bills and continue to support Travis CI as an open and
free service for the Open Source community. But in order to have some more free
resources for this we will also look into follwing up on the crowdfunding
campaign for the next year. Expect to hear more about this soon.
So, if 2011 was the year of birth of the Travis CI project and 2012 was the
year of its childhood and youth, we hope that we can make 2013 the year of
growing Travis CI up to stabilize as the most useful, efficient and well
supported continuous integration platform out there.
We are super excited and looking forward to working with you and we are
extremely thankful to be able to be part of an amazing project and a fantastic
community like this.
Thank you all for a fantastic year!
2012 was a blast. Thank you so much.
Love you all :)
