Usability Testing: 3 Ways to Take Your Product’s Pulse

Your company has just released a new website. After months of development and hard work, the team has made the perfect final version and finally roled it out into the beta stage, with real people are using the site. But there seems to be a disconnect. The traffic is there, but you just aren’t generating as many conversions as you originally expected to. So what gives?

It’s hard to tell what elements work effectively and which ones slow users down. Your team members can’t make a good judgment either, as you’ve all played a part in putting the product together and won’t look at it the same way a stranger would. What to do? The answer lies in usability testing.

Remember, usable does not mean functional. Just because you’ve designed something and it works does not mean that the user will figure out how to use it.

Many have the perception that usability testing is outdated, costs to much money to run, and takes up too much precious time. Dana Chisnell, the writer of an entire blog dedicated to the subject puts it best when she says, “Conducting dry runs, practice sessions, pilots, and rehearsals — call them whatever works in your team — gives you the superpower of confidence. That confidence gives you focus and relaxation in the session so you can open your mind and perception to what is happening with the user rather than focusing on managing the session.”

While the concept is simple, to have consumers interact with the product and track their usage to see where they fall out along the way, there are so many tools available that it can be difficult to know where to begin. Joe Ayotte came to the rescue on Monday night in his class on the same subject. Joe showed us the ropes of the usability testing universe, provided some great tips and tricks, and proved that it actually doesn’t cost as much (if any!) money as one would think. He broke the process down into three simple areas.

Feedback from Users

Perhaps the cheapest and easiest way to go about usability testing is to gather qualitative data through feedback from people who have used your product. It can be as simple as emailing a user and asking what worked for them and what didn’t? Why didn’t they complete their transaction? Other options include in-person observation while the user plays around on the site, noting where they stuck. Answers to these questions can provide valuable insights into the users’ minds.

Does this sound like a lot of work? It doesn’t have to be. There is a host of tools available that can automate the process for you, like

  • Userlytics – Watch and listen to participants, from all over the world, as they conduct tasks, answer questions, react to stimuli and interact with your prototype or production assets.
  • – Similar to Userlytics, you can watch videos of participants using the site, receive written answers to questionnaires, and interact with users after the test to ask follow-up questions.
  • OpenHallway – Create scenarios for participants and receive visual feedback directly from the participant.

A/B Testing

In addition to qualitative data, you’ll probably want some hard quantitative data to go along with it. A great process to gather quantitative data can be found in A/B testing. As Eric Hansen told in a recent interview, “A/B testing is probably one of the most graspable and effective ways to apply “the scientific method” towards improving your web site. Usually it comes down to presenting multiple versions of your site (or an individual element on a site, like an image or a promotion) to end users, and then measuring behavior against some business goals such as conversion rate. I often compare A/B testing to a clinical trial, only instead of doctors, patients and drug treatments, it’s marketers, end users and site variations. A/B testing is easy to understand, and done correctly, can provide a really compelling way to iteratively improve your web site over time.”

Tools like Optimizely, A/Bingo, which is open source, and even Eric’s own SiteSpect> make great options to A/B test your site.

Website Analytics

Analytics tools provide another way to gather quantitative data. There is a smorgasbord of services out there that measure the fine details of how your customers interact with your website, including

  • ClickTale – Track users’ interaction with your site. They provide heatmaps to measure mouse movements and clicks, in addition to conversion funnels and form analytics.
  • Usabilla – Measure participants emotions, couple them with data trends, and find new ways to make improvements to your site.
  • Loop11 – Gather insight into usage stats through clickstream analyses, heatmaps, and video playback.

With a full set of these tools at your disposal, you’ll soon be able to optimize your site and finally convert those users. For more great lessons, check out the blog, or visit our class page.

Photoshop with Ash Edmonds: Tool or Toy?

Photoshop. You know it well, but do you know how to use it? From working in design to marketing, product or sales, Photoshop can be a useful tool for helping you find your creative side, and surprisingly, it’s not that difficult to get the hang of. Whether your goal is to develop a logo for your new company or tap into a new source of expression for your own free time and amusement, your first stop should be the Photoshop 101 class in Boston with Ash Edmonds, the Head of Design at Dailybreak on Monday, July 23 at 6:00 p.m. at!

Ash Edmonds Photoshop

What’s the most bizarre photo you’ve ever edited / created in Photoshop?

The most bizarre photo I’ve ever edited was of my friends and I in Germany. The edit was primarily focused around lighting, using a photography technique called HDR, or High Dynamic Range, which merges multiple exposures of one photo together. For this photo I took the original, two under exposed versions, two over exposed versions and mashed them together. As a result, a whole new set of lightening and detail possibilities are available.

How might someone who’s not a designer put Photoshop to work?

Even if you’re not a designer, Photoshop is a very valuable skill to have. It’s much more mainstream and odds are almost everyone knows what Photoshop is. A content writer, developer, or a co-worker in marketing might work with a designer everyday and could benefit by knowing what goes into a particular design and how much time it might take. A smaller piece of the design project may be delegated to someone from another department to make the process more efficient. The writer can easily crop a photo, add text to it, or make it black and white. A developer might take a UI design within Photoshop and by understanding layers will be able to export all the necessary images. A co-worker from marketing might need a design for the companies Facebook page. By understanding Photoshop that person can take a designers file and re-create it in a different size.

If you could be an expert in one skill by tomorrow, what would it be?

If I could be an expert in one skill by tomorrow I would be a master at compositing. Compositing is the total creation of a scene, bringing aspects from other photos, creating custom shapes, and seamlessly integrating all elements with respect to shadows and lighting to make a realistically designed environment. Being a master at compositing requires a keen eye, dexterous skill, and the patients to craft with attention to detail.

Ash EmondsAsh Edmonds got his start as an Interactive Designer at the Staples HQ then after 3 years, joined the startup community with Dailybreak as Web Designer. He shortly moved on to build a super talented design team as Head of Design. Ash is a Photoshop guru, Photographer,and friend of

Ash is teaching Photoshop 101 at on Monday, July 23rd at 6:00 p.m.

The Ultimate Landing Page Guide

The campus recently welcomed back Christopher O’Donnell for an encore appearance following his May class around product management. You better believe he delivered big again, this time with The Ultimate Landing Page Workshop!

Types of Marketing MessagingThis class was heavily example-driven, and students were even given the opportunity prior to the class to send in their own landing pages to be judged. For some it was a pedestal. For others, a guillotine. One landing page in particular had several flaws that Christopher was quick to correct and offer advice on—good thing they came to class…

Having a successful landing page is all about give and take. The ultimate goal is obviously to secure and retain a customer. However, the smaller, sometimes underestimated exchanges in between is what can make the difference between building a customer base and chasing your customers away.

Deliver Value

Value. It’s one of the few things in this world that everyone wants. Valuable commodities, valuable information, valuable time. Your potential customers are no different. What does it take to make someone a lead? Simple. Offer them something of value. It might even be as bartering an eBook for an email address. Make them feel as though they’re getting something for not much at all. Once you get that small piece of contact info you’ve officially gotten a lead, which, with plenty of water and sunlight, should eventually become a customer. All you’ve given up is an eBook fresh off the press.

Peg Your Personas

Tip: Along with the email, ask “What best describes you?” to help peg your personas. Provide users with descriptive, straight-forward options in a dropdown box. This simple step allows your leads to organize themselves for you.

Experiment with Tone

There are four basic types of messaging you can employ in your landing page: Fear-based, Descriptive, Promise-based, and Social Proof. Here are some examples of each.

  • Fear-based: “Don’t let this happen to you!”
  • Descriptive: “Light, fluffy, and straight from the oven.”
  • Promise-based: “You’ll see your waist shrink four sizes, guaranteed.”
  • Social Proof: “This class helped me get a promotion!”

Switch Your Lense

Sometimes seeing your landing page through your visitors’ eyes is the best way to understand what you’re doing right and where you’re losing people. Some things to consider:

  • How did they get there?
  • What is their state of mind?
  • How well do they know you?
  • How well do they know what you’re offering?
  • How motivated are they?

This is just a taste of what Christopher offered in class. For access to the slides he used in class, click here.

Eric Hansen on A/B Testing

Imagine for a second (or maybe you don’t have to imagine) that you’re the CMO of a new company and you’ve charged your team with the task of coming up with some concepts for your new site design. Your team comes back to you and you choose the best two, but they’re both really good. Both of them work really well with the message you’re trying to get across and neither member of your team is willing to give up on their design. So what do you do? A/B test it of course!

Eric Hansen (@ericjhansen) of SiteSpect will give a crash course at on June 13 on Web Optimization 101: Improving Your Site Through A/B Testing. It’s a simple concept, but Eric dropped a line and explained what his company does and how the tool can really help.

I.LY: Tell us about your company SiteSpect and what your role is as founder and CEO.

EH: SiteSpect provides A/B and multivariate testing (among other things) to companies of all shapes and sizes.  We’re both a technology (PaaS) and a services company, so not only do we provide the tools, but for companies that need help we also provide consulting and hands-on guidance.  My role as founder and CEO is to steer the proverbial ship, growing our business and taking very deliberate steps to ensure that our customers get the best overall value from working with us.  On a day to day basis, that means working with our product teams to evolve our core technology (I’m also the de facto CTO), and helping our VP’s to achieve success in each area of their business areas.

I.LY: For those that may have never heard of it, what is A/B testing?

EH: A/B testing is probably one of the most graspable and effective ways to apply “the scientific method” towards improving your web site.  Usually it comes down to presenting multiple versions of your site (or an individual element on a site, like an image or a promotion) to end users, and then measuring behavior against some business goals such as conversion rate.  I often compare A/B testing to a clinical trial, only instead of doctors, patients and drug treatments, it’s marketers, end users and site variations.  A/B testing is easy to understand, and done correctly, can provide a really compelling way to iteratively improve your web site over time.

I.LY: Has the core of your service changed over the years since the advent of Web 2.0 and social media?

EH: One of the biggest changes has simply been around breadth and scale.  Whereas in 2004 we were handling testing and optimization for core HTML content, today we’re also doing client-side script optimization (e.g. JavaScript and CSS), web application optimization (e.g. server-side functionality), and even mobile web optimization.  And, we now handle many of the world’s largest sites on our technology platform, which means massive volumes of transactions and data that we need to crunch and analyze in real time.  It’s a challenge, but it’s also what keeps things so interesting for us on a daily basis!

I.LY: How do you see the web evolving in the future?

EH: More ways for users to interact (both with brands and with each others), on more devices, in more places, across more contexts.  “Digital ubiquity” is the term, I think.  As all of this happens, each new inflection point presents another greenfield of opportunity for digital marketers and analysts who strive to provide their audience with the best possible user experience.  If there was ever a case where status quo = falling behind, this is it.  Where will we be in a few years?  Mobile, TV (set top), digital billboards, heads up displays, holographic content, dynamic skywriting.  (Ok, I made that last one up.)  One thing for sure, it’ll be exciting.


You can optimize your site too! Sign up now!

Web Optimization 101: Improving Your Site Through A/B Testing
Wednesday, June 13, 2012
6:00 – 7:30 pm
500 Harrison Ave, 3R
Boston, MA 02118
Sign up here!