Adding 4000 subscribers in 45 days

I love researching things. Learning about something new and finding the best way to approach a problem feeds the engineering part of my background. I satisfy that by reading a ton and because of my online businesses, I read a lot of SaaS-related online marketing and sales material. But sometimes I read more than I act. As a recent example, in setting up the best possible go-to-market strategy for our new live timing app, RaceHero, I ran out of time before vacation and lost weeks out of my marketing strategy by failing to kick it off before I left town. That same “cobbler’s shoes” fate had befallen my email lists for the past 9 years.

This year I’ve been working to do more A/B testing. A/B tests are mathematically-backed competitions between two or more options which are scored by the actions of the users. They may also be the single highest ROI tactic for software companies. These tests could be anything from a simple color change of a button (shocking, to what extent that can matter) all the way to a completely different web page. I was first exposed to the technique back in 2001 at Yahoo! when I was helping redesign their search results but it wasn’t until the last 18 months I ran the first A/B test on MotorsportReg.com.

Which brings me to how I grew one of my mailing lists by 53% in 45 days. In the history of this 9-year old list, the last two months would make a hockey stick look like a rolling foothill. This is like the face of a glacier where it meets into the ocean: a vertical line that reads like an error.

The technique is perhaps too simple to be valuable by itself: I put the signup form where many more people would see it in their routine use of our platform. Previously, we had a few CTAs in our transactional emails and on our event calendar but the actual signup took place on a mailing lists screen under “My Account”. Few people went there looking to add more email to their inbox so list growth was steady but slow. Now, when users create a new account or reconfirm their details, they see checkboxes at the bottom of their personal information letting them opt-in to the lists. Thousands of people see these screens every month and because the lists are valuable, thousands of them are signing on.

“It was just a test”

The obvious question is why didn’t we do this sooner? The answer is because I wasn’t sure if these transactional flows were appropriate places to insert a list signup. I had concerns that it might negatively impact registration conversion. So I hemmed. And I hawed. And I periodically looked at the issue in our bug tracker but never quite got over the “ewwwww” feeling to put it in place.

That’s the beauty of A/B testing. A feature request in our bug tracker is a stone being added to a wall with mortar: it becomes a seemingly permanent piece of your architecture, of your user interface and of your responsibility. But an A/B test, why, that’s just a test! We have made no commitments to keeping it around. We’re not even sure we like it! Implementation only took a few hours so we could rip it out at any time for any reason and respond, “It was just a test.” Simply doing it, rather than talking about it, is rule #2 of A/B testing fight club.

Since it was just a test, we hoped for a nice bump to justify keeping it around. We were blown away. At the current rate, we will reach 25,000 subscribers (333% growth) by the end of 2015. And because we have a large percentage of first-time participants come through our service, we should see a permanently scaled growth rate. As an additional bonus, this list drives participation for our events so we’re creating a positive feedback loop for our event organizers and our bottom line too.

All because of a feature I was skeptical of “implementing” but happy to “test”.

Remember: making the call is making progress. Doing is better than planning. Execution is more valuable than ideas. Look at your to-do list and find one or two things you’ve been putting off. What can you do to “test” it (whatever that might mean in your case) to move it ahead and gain confidence in your choice?