In private projects, such as the ones our customers are running on Travis Pro, a large integration test suite is a common part of the build process.

As these integration test suites tend to test more complex scenarios through the entire stack, they also tend to be the slowest part, requiring multiple minutes to run, sometimes even up to half an hour.

To speed up a test suite like that, you can break it up into several parts using Travis’ build matrix feature.

Say you want to split up your unit tests and your integration tests into two different build jobs. They’ll run in parallel and fully utilize the available build capacity for your account.

Here’s an example on how to utilize this feature in your .travis.yml:

env:
  - TEST_SUITE=units
  - TEST_SUITE=integration

Then you change your script command to use the new environment variable to determine the script to run.

script: "bundle exec rake test:$TEST_SUITE"

Travis will determine the build matrix based on the environment variables and schedule two builds to run.

The neat part about this setup is that the unit test suite is usually going to be done before the integration test suite, giving you a faster visual feedback on the basic test coverage.

Depending on the size and complexity of your test suite you can split it up even further. You could separate different concerns for integration tests into different subfolders and run them in separate stages of a build matrix.

env:
  - TESTFOLDER=integration/user
  - TESTFOLDER=integration/shopping_cart
  - TESTFOLDER=integration/payments
  - TESTFOLDER=units

Then you can adjust your script command to run rspec for every subfolder:

script: "bundle exec rspec $TESTFOLDER"

For instance, the Rails project uses the build matrix feature to create separate jobs for every database to test against, and also to split up the tests by concern. One set runs tests only for the railties, another one for actionpack, actionmailer, activesupport, and a whole bunch of sets runs the activerecord tests against multiple databases. See their .travis.yml file for more examples.

You asked for it, so here it is.

Today we are happy to announce that we have shipped the new Travis CI web client for Travis Pro.

You can find a link to the new client on Travis Pro, we’ve sent a broadcast to everyone.

Travis Pro is still in beta. If you are interested in trying it out but don’t have an account then drop us an email at support@travis-ci.com.

We are going to follow the same practice that we’ve followed shipping the client for .org: We’ll make the new client available on a separate domain, in parallel to the old client. This way we can ensure there’s always a working, stable way to access Travis Pro, even if the new client should show any issues. We’ll keep things this way until we are sure everything is fine and then finally retire the old client.

Please help us by giving feedback on any issues, breakage or weirdness.

If you are eager to read an overview of all the features and goodies that the new UI includes you can find them in the original announcement. But the gist is: overall polished, much more responsive, fast and stable, new features like proper URLs, requeueing builds, flash messages etc.

A lot of people have asked us about our pricing for Travis for private repositories, and it’s about time we lift the curtain and be open about it.

A little note up-front. We made a rookie mistake while working towards finalizing billing and letting everyone currently on the platform know about enabling it:

We unfortunately didn’t collect email addresses properly. We only get email addresses from GitHub when you publicly share your email address.

So we had to include this blog post in the build emails to announce our plans to transition Travis for private repositories from a private beta to a paid private beta.

A Little History

While the first build on Travis for private repositories was done in April, we officially invited people to use it in June, ramping things up in July. By now we have close to 200 organizations actively using it.

We’d like to say thank you to everyone who’s put their trust in us, tried us out, and provided valuable feedback during the last four months. In that time, we’ve done more than 140,000 private builds on our infrastructure, with some of our early customers pushing Travis to the limits, showing us the places we need to improve, which benefits both platforms in the long term.

We’re now taking the next step, and after almost five months of providing free service, we’re moving Travis for private repositories to a paid private beta. We would love to open it up for everyone, but to anticipate capacity, we’re keeping Travis for private repositories in private beta for a bit longer. We want to provide the best continuous integration service for everyone once we open it up.

You might be wondering why we’re writing about our pricing in a blog post instead of having a proper product page for Travis. It’s something we’re actively working on, and we wanted to be very open about our subscription plans as early as possible.

What Existing Customers Are Saying

We’ve slowly started rolling out billing three weeks ago, and we’re happy to report that we already have a humbling number of customers who’ve put their trust in us, with a few of them listed below.

We’re very grateful for this and would like to say thank you to these companies! Thank you for putting your trust and money in Travis, and for allowing us to continue improving both Travis platforms.

Thank you, Thunderbolt Labs, Nedap, Wooga, SEOmoz, Flying Sphinx, Readmill, LessEverything, Code Miner 42, DNSimple, Kisko Labs, BusyConf, Librato, Apcera, Moovweb, Flinc, Gidsy, Envy Labs, Hypertiny, 6wunderkinder, 20minutes, Crowd Interactive, 8th Light, Intridea, Spree Commerce, Amen, and many more!

We love you!

Here’s what some of our customers say about Travis for private repositories.

Tammer Saleh, Thunderbolt Labs

Any development team that cares about quality needs to be using Travis CI.

Eric Allam, Envy Labs

Travis CI is the CI system I’ve always wanted, dead simple and with great Github integration. The bottom line is that it’s improved our application and our confidence in rapidly shipping code.

Micah Martin - Founder and CTO at 8th Light, Inc.

CI is hard. Travis-CI is effortless. Signing up for Travis-CI is a no-brainer.

Hampton Catlin, Moovweb

Travis makes my life so much easier with its Github integration. Now, everyone on our team makes sure their pull request passes tests before submitting it, saving me a ton of time and energy.

Sean Schofield, CEO Spree Commerce

Travis helps keep things running here at Spree Commerce. Everybody needs tests but nobody wants to run their own CI server. Bonus points for awesome Github integration!

Paul Campbell, Hypertiny

Travis gives me an immediate answer to : can I deploy now? can I merge now? Travis is painless to set up, and it just works. It’s a simple, clean execution, solving a problem where no easy do-it-yourself option exists. Top class.

Arnaud Limbourg, 20minutes

Great tool that works and good customer service, off to a good start I say

David Pavilla, Crowd Interactive

Travis changed the way we did CI since day one. As a 100% TDD company, it is very important for us to always know if a build is good or bad, specially because there’s always at least 4 people working on the same codebase. Now, we can set up a new projects real quick and without worrying too much on the details of the implementation. For our open source projects we’re saving a lot of time too, now that it automatically tests all pull requests. I can’t wait to see the features to come in the future that will make it even more awesome.

Ryan McGeary, BusyConf

Travis-CI’s GitHub integration is amazing! Travis-Ci is like having a little robot follow you around making sure you didn’t miss anything while building that next feature or expirimenting on a new branch of development. Travis accelerates our development at BusyConf.

Derek Collison, Apcera

Continuous Integration and Testing is key to building the reliable distributed systems that Apcera is designing and architecting. We are very happy with our choice of Travis-CI, the team has been responsive at every step. Happy to be a customer!

Anthony Eden, DNSimple

There are few things more frustrating than setting up and maintaining your own continuous integration system, and yet having a CI is an essential part of any software development process. Fortunately TravisCI’s new pro offerring makes CI painless. I am thrilled to have the opportunity to support the Travis CI team as they continue to improve both the free open source and pro offerings. As an added benefit, pro members have the option of getting ridden into the sea like a wild unicorn by Travis CI’s very own Josh Kalderimis, an offer that is too valuable to pass up.

Benedikt Deicke, flinc

Since switching to Travis Pro, we stopped worrying about our CI setup: It just works and it’s better than anything we had before.

Steven Bristol, LessEverything

We tried a few a few CI solutions before trying to Travis. Nothing is faster, better, or easier. Plus the people that work on it are great! We couldn’t be happier.

Philipp Wassibauer, Gidsy

Travis CI has increased the speed at which we release new features here at Gidsy. The tight integration with Github - especially seeing the test status of each commit within a Github Pull Request - and an interface that is not stuck in the 90’s makes our team more productive. But it might also have something to do with the Travis CI founders coming by our office and bringing us fresh coffee!

We love the product and can’t wait to see how it develops.

The Subscription Plans

Billing on Travis for private repositories is done on a monthly basis per GitHub organization or user account using Travis. Every plan includes unlimited builds, unlimited repositories, and unlimited collaborators.

As developers, we want you to focus on building and shipping great apps instead of counting build numbers or build minutes. So we’re removing any artificial boundaries right from the start.

Our small plan allows two builds to run at the same time. While you can push as much code as you like or have your build expand into lots of smaller steps, only two of them can run concurrently. Our bigger plan allows you to run five concurrent builds.

Our small plan is $129 per month, and our big plan is $249 per month. We’re happy to provide more build capacity if required, please get in touch if you need more than five concurrent builds!

Note that these prices are only for private repositories. Travis for open source projects will always be free to use for everyone!

The subscription plans not only allow us to provide the best platform for testing private repositories, it also makes sure we can continue supporting and maintaining Travis for the open source community!

What’s next?

The subscriptions and plans are available for everyone using the platform as of today. You’re welcome to continue using the beta platform for free, please be aware that we’re putting a cap on free usage in two weeks, effective from 7th November 2012. Please make sure you’ve signed up for a subscription to continue to use our service by then.

To sign up, log into Travis, go to your profile, and select the organization you’d like to sign up for a subscription. Enter billing and credit card details, and done!

If you have any questions, please get in touch! If you want access to the private beta, please get in touch and we’ll see what we can do for you!

If you’ve signed up for our waiting list or donated to our love campaign, we’ll be in touch with a special treat soon!

We’re sorry that we have to postpone opening up Travis for private repositories for everyone just a bit longer. We can’t wait to open it up for everyone soon!

But what’s next for Travis?

We’re working on lots of things and have a long list of things we’d like to ship. First and foremost, we’re working hard on getting our new web interface out the door for Private Travis, which also includes our new API. More on that soon.

We have several things in the works for better virtualization. Moving off VirtualBox is one of our main goals for the coming months. Expect a blog post with more details soon.