We’ve been working on turning Travis CI into a business for more than a year
now. While we’re working hard to go public for everyone to use our platform for
private repositories, there have been quite a few books that have inspired us.
You’ll notice that none of them relates to code or continuous integration, but
that’s a good thing. Building a product involves only so much code, but it
involves a lot more of marketing, customer support, improving the user
experience, finding the right price, and a lot more.
I’ve been reading a lot of books to give me food for thought in these
directions, and I wanted to share some of them.
The Knack deals a lot with the every day of running a business. While a lot of
it is drawn from running bike courier and box storage companies, there’s a lot
of inspiration to draw for online businesses like ours, in particular when it
comes to numbers (finding the right price, margins, and so on) and handling
customers. A highly recommended and easily approachable read.
Finding the right price for a product is never an easy task. It certainly has
cost us months to get to a point where we’re comfortable, and it’s thanks to
this book.
If you need to find outside reasoning on how to find the right price for your
product, this book is for you.
I consider this the little brother of “Pricing With Confidence”. It’s a short
read on finding the right price, and will make you want to know more about
motivations behind higher prices, value addons, and all that.
A classic in its own right, dating back to the 1930s, but still very relevant
when it comes to dealing with other people.
Customer support involves people approaching you in all kinds of mood, and it’s
the art of turning every customer around by treating them right. This book was
an eye-opener for me in this regard.
It’s a must-read not just when you’re looking at running your own business, but
to help you deal with all kinds of people every day.
This is an incredible book, full of empirical proof on how humans decide on
things. It starts out with a great bang that’ll help you structure your pricing
right, gives you inspiration on how you can utilize free stuff to sell your
product, and a lot more.
An inspiration read, full of good stuff. Best of all, it’s all backed up by
studies to give you a great insight on how humans decide on things.
A book full of good examples on running a profitable small business without
feeling the need to go for insane growth, but rather to go for a profitable
product, how to market it, and how to find the right audience. An easy but great
read.
A great exploration of human habits and how companies utilize them to benefit
their own business and how they shape our every day. It may not seem about
business on the surface, but everything in this book is very applicable for
turning your customers into loyal customers and making them happy.
This book has taught us a lot on how we talk to customers, and how we compose
blog posts, marketing messages, product information, and even how we write our
post-mortems.
It’s about dropping ambiguous words from what you say to other people and be
more to the point. Instead of saying no, don’t or anything negative, it’s about
focusing on positive outcome, on focusing what you want to achieve.
Most importantly, this book is about honesty, an honorable trait in any
business.
After a bit of a delay, we are happy to let everyone know that our VMs have been updated with a host of fixes, updates, and general polish and shine. From language versions, to services, to system settings.
These VM changes will be live for https://travis-ci.org users tomorrow (Tuesday 18 June, 12pm CEST), and for https://travis-ci.com users on Monday next week (24th June, 12pm CEST).
If you have any issues or requests, please create an issue on our issue tracker.
Below are the full details of the updates included in the VM refresh.
Have a great Monday,
Josh and the Travis Team.
Language Updates:
Ruby:
JRuby 1.7.4
MRI 1.9.3-p429 and p327, and 2.0.0-p195
Node.js:
0.8.24, 0.10.9 and 0.11.2
Go:
1.1.1 preinstalled and set as the default
PHP:
5.3.26, 5.4.16 and 5.5.0RC3
Perl:
added 5.8, 5.18 and 5.19
updated current versions to 5.12.5, 5.14.4, and 5.16.3
Python:
latest patch versions installed
Services Updates:
Redis: v2.6.13
ElasticSearch: v0.90.1
MongoDB: v2.4.4
Cassandra: v1.2.5
PhantomJs: v1.9.1
Note worthy mentions:
Clojure: Leinigen updated to v2.2
Composer: Read-only GitHub OAuth token added, this should help resolve rate limit issues.
PHP: PEAR is now available on v5.5
Cassandra: Config fixed, service should now start up like a dream.
Postgres: shmmax and shmmall both increased
Tmpfs for Mysql and Postgres DBs doubled to 512megs
Mirrors removed: We provided EU based mirrors for Maven and Perl, these have been removed as our VMs are based in the US.
FireFox locked to 19.0, see http://about.travis-ci.org/docs/user/addons/#Firefox for more info
Travis CI and continuous integration are all about making sure your code is
fulfilling the expectations set against it by way of your tests, whether
they be unit tests, functional tests, or integration tests.
Moving into the realm of continuous delivery though, you get to a simple
question: How can you ensure that your code does what it’s expected to do when
running in production?
When code runs in production, it doesn’t have the protection of a clean
environment like your tests do. It’s suddenly exposed to the harsh reality of
life in production and therefore to random influences and emerging situations,
like network timeouts, slow disks, errors in third-party APIs.
As you ship code more often to production, you need to ensure that your code is
doing what’s expected of it. You need automated ways of collecting data and you
need automated ways to make them easy to analyze.
You need metrics, and you need logs.
Metrics: The What
Metrics allow you to slap a number on anything, to measure how often something
is run, how long it’s taking, how much memory a process is consuming, how long a
page took to render, how long a database query took, how many external API calls
have failed, how many errors your application has been raising, how many failed
logins you’ve had on your site.
In short, anything that moves. Anything that’s indicative of potential problems
on the site or that’ll help you investigate when problems occur. Rest assured,
they will occur.
Metrics allow you to stay up-to-date on what’s going on inside of your
application, but on a numerical level. Depending on the tools you use, you’ll
also get nice ways to visualize the data you collect over time.
Here’s a list of things that we’re tracking in a normal workflow of a build in
Travis CI:
Request times accessing the GitHub API to fetch the .travis.yml
The number of GitHub notifications we handle that get turned into a build
The time it takes from a build being created until it’s scheduled to run
The number of currently running builds
The time since the last build was started
The time since the last build has finished
The time it took for the build environment to boot
The number of email notifications we send
This is just a sample, but there’s a lot going on in Travis CI. Here’s an
example, the number of builds currently running on http://travis-ci.org.
What’s more important, it’s very easy to add new metrics and have them be
collected in our collection system. We use Librato Metrics
to store and visualize our data, and we’re using Eric Lindvall’s excellent
metriks library.
Making it easy for developers to add and visualize new metrics is just as
important as enabling developers to build and continuously grow a test suite.
The easier it is for someone to add a new metric and see it pop up somewhere,
the more likely the buy-in from your team to get started.
I don’t have any metrics, get me started!
With a legacy code base (one that doesn’t have any metrics) similar ideas apply
as to how to sneak tests into projects that don’t have any yet.
You start small and simple by adding metrics to the parts of the code you’re
currently working on. You can gradually work your way up, that’s how the Travis
CI code base got instrumented over time.
For instance, if you added a new feature that sends password reminders to users.
You can start tracking how many you’re sending by adding a single line to your
code.
def send_password_reminder(user)
Metriks.meter('password_reminders').mark
UserMailer.password_reminder(user).deliver
end
This tracks how many you send. You can keep going and instrument the sending
itself. Most of the time, when sending emails, some external resource is
involved. Whether you’re talking to an API (think Mandrill, Postmark, et. al.)
or using SMTP directly, there’s network overhead worth measuring.
def send_password_reminder(user)
Metriks.timer('password_reminders').time do
UserMailer.password_reminder(user).deliver
end
end
Lots of libraries exist to instruments for all kinds of languages and
frameworks. Most of them have been inspired in one way or the other by Coda
Hale’s metrics library, and the talk it’s
based on, “Metrics, Metrics
Everywhere”. The talk
gives a great introduction on the types of metrics that you can use to
instrument your code, why you should instrument, and how that improves the
transparency and visibility of what your application is doing.
The upshot of well-put metrics and accessible visualization is that you can put
them up on a dashboard to increase visibility for everyone in the company, for
everyone with a stake in the product.
Logs: The Why and the How
Where metrics tell you all about what’s going on, logs are the true soul of your
application. Metrics only give you a numeric representation of what happened, logs
tell you (ideally) why things happened. They’re your audit trail of steps your
application took to execute a specific task.
While metrics can be easily unified using a common library, for logs that’s much
harder to do. We’re using part automated logs that give a simple audit trail and
part literal log output to determine what happened.
It can be tempting to try and make logging for your application generic. This
works for quite a few things (tracing method calls, HTTP request
details).
The important thing to remember is, when there’s an incident that requires
investigation, there’s a person looking at the logs, trying to make sense of
what happened and why.
Logging is the art of not saying too much, but also saying enough that an
investigation can be done by a human. Just like adding metrics, it’s an
evolutionary process, there’s no one right way to do it and to find the spots in
your application that need to log meaningful data.
For instance, the part of our apps that handles trials for http://travisci.com
logs statements about trials that have expired, trials that have started and all
that. On top of that we track metrics, but they’re anonymous, they have no
relation to the account the trial is running for.
Once you’ve started logging, you need a place to aggregate them for simple
search. There are services to help you get started, we’re using
Papertrail for this very purpose. It’s wonderfully
simple to hook it up to your Heroku apps and start logging.
Measure Everything, Log Anything!
To make sure that you can continuously ship code to production and to have
confidence in what your application doing when running, logs and metrics are
your first steps to more transparency and visibility.
While there’s no one way to integrate them, there are certainly a lot of options
to get you started. Combined with a healthy test suite, both allow you to build
and ship your code with confidence.
