Eric Hansen on A/B Testing

Imagine for a second (or maybe you don’t have to imagine) that you’re the CMO of a new company and you’ve charged your team with the task of coming up with some concepts for your new site design. Your team comes back to you and you choose the best two, but they’re both really good. Both of them work really well with the message you’re trying to get across and neither member of your team is willing to give up on their design. So what do you do? A/B test it of course!

Eric Hansen (@ericjhansen) of SiteSpect will give a crash course at on June 13 on Web Optimization 101: Improving Your Site Through A/B Testing. It’s a simple concept, but Eric dropped a line and explained what his company does and how the tool can really help.

I.LY: Tell us about your company SiteSpect and what your role is as founder and CEO.

EH: SiteSpect provides A/B and multivariate testing (among other things) to companies of all shapes and sizes.  We’re both a technology (PaaS) and a services company, so not only do we provide the tools, but for companies that need help we also provide consulting and hands-on guidance.  My role as founder and CEO is to steer the proverbial ship, growing our business and taking very deliberate steps to ensure that our customers get the best overall value from working with us.  On a day to day basis, that means working with our product teams to evolve our core technology (I’m also the de facto CTO), and helping our VP’s to achieve success in each area of their business areas.

I.LY: For those that may have never heard of it, what is A/B testing?

EH: A/B testing is probably one of the most graspable and effective ways to apply “the scientific method” towards improving your web site.  Usually it comes down to presenting multiple versions of your site (or an individual element on a site, like an image or a promotion) to end users, and then measuring behavior against some business goals such as conversion rate.  I often compare A/B testing to a clinical trial, only instead of doctors, patients and drug treatments, it’s marketers, end users and site variations.  A/B testing is easy to understand, and done correctly, can provide a really compelling way to iteratively improve your web site over time.

I.LY: Has the core of your service changed over the years since the advent of Web 2.0 and social media?

EH: One of the biggest changes has simply been around breadth and scale.  Whereas in 2004 we were handling testing and optimization for core HTML content, today we’re also doing client-side script optimization (e.g. JavaScript and CSS), web application optimization (e.g. server-side functionality), and even mobile web optimization.  And, we now handle many of the world’s largest sites on our technology platform, which means massive volumes of transactions and data that we need to crunch and analyze in real time.  It’s a challenge, but it’s also what keeps things so interesting for us on a daily basis!

I.LY: How do you see the web evolving in the future?

EH: More ways for users to interact (both with brands and with each others), on more devices, in more places, across more contexts.  “Digital ubiquity” is the term, I think.  As all of this happens, each new inflection point presents another greenfield of opportunity for digital marketers and analysts who strive to provide their audience with the best possible user experience.  If there was ever a case where status quo = falling behind, this is it.  Where will we be in a few years?  Mobile, TV (set top), digital billboards, heads up displays, holographic content, dynamic skywriting.  (Ok, I made that last one up.)  One thing for sure, it’ll be exciting.


You can optimize your site too! Sign up now!

Web Optimization 101: Improving Your Site Through A/B Testing
Wednesday, June 13, 2012
6:00 – 7:30 pm
500 Harrison Ave, 3R
Boston, MA 02118
Sign up here!


One thought on “Eric Hansen on A/B Testing

  1. Pingback: Usability Testing: 3 Ways to Take Your Products Pulse -

Leave a Reply

Your email address will not be published. Required fields are marked *