Scoop

How many times has your designer kicked out a ‘final’ creative asset based on best practices, gut feelings, “experience”, or if you’re lucky, historical data? Ever get the sense they nailed it on the first try? I haven’t and it can be so frustrating to ask for multiple versions or styles. Well, when resources are constrained, the best thing to do is utilize your knowledge of A/B and Multivariate testing.

 Testing Methods

There are several methods to employ when testing components of your creative assets, but the two most common are; A/B and Multivariate testing. For the sake of simplicity we will focus on these two and leave the more complex testing methods to the professionals.

 A/B Testing

A/B testing is defined as the method of testing a baseline Control [original creative asset] compared to a single-variable Sample [your revised asset] to improve response rates. Making sure you only have one variable being tested during the process is critical to success. Skewing outcomes can be very easy when introducing multiple iterations if you’re not controlling for them.

A simple example, if you’re A/B testing a subject line in an email (Control and Sample) to see which has the higher open rate, keep it as simple as possible. Send out the emails at the same time, don’t change email headlines or CTA buttons and only test for subject line. If you introduce other variables, you will skew your results. Alternatively, if you’re testing the color of a call-to-action button, you will want to keep everything static on the page, only test one variable at a time; your call-to-action button.

I like having several versions of an asset queued up to test. Use the existing asset in place as your Control and have several different Samples in the queue ready to rotate and test. Test your Control to Sample 1, if the Control wins, test the Control against Sample 2. If Sample 2 wins, test Sample 2 against Sample 3, etc. I call this simple A/B progression testing.

Controlling for variables outside of your website or design is important. For example, control for; large social events (the 2008 Olympics skewed tests I ran during that time), time of day, day of week/month, and most importantly, any seasonality your business experiences throughout the year.

 Multivariate Testing

When looking to test more than one variable at once, you’ll want to engagement the methods of multivariate testing. Defined as the process by which more than one component of a website (or creative asset) may be tested in a live environment. For example, perhaps you have five different hero images and five different headlines you’re interested in seeing which combination yields the highest click-through rate. Ensure you have the right technology to randomize the 5 different hero images and headlines evenly across time to ensure a large enough sample size interacted with your test before drawing conclusions.

Another form of testing multiple variables is what I like to call multilevel-multivariate testing. An email example of this includes; testing different subject lines, the color of your email’s call-to-action button, and the copy of your Sign Up page’s “submit” button. This form of testing involves three assets and flows through three different interaction levels; 1) opening the email (subject line), 2) clicking-through from email to website (call-to-action button) and 3) conversions on your website’s landing page (submit button).

 Testing Sample Size

I can’t tell you how many times I’ve rolled out a simple test and got so excited when I saw my Sample was winning. I would run around and tell everyone the new design was winning and yell and scream we should roll it out 100%. Well, thankfully there were others there to tell me to shut up, calm down and check my sample count. You can’t make rash decisions based on premature data and sometimes need an impartial party to keep you in check.

There are a few components you need to understand prior to testing; sample size, confidence level, confidence interval and population.

Sample Size:

• These are the users exposed to your test, the more users within your sample, the greater the likelihood your results will represent your entire population.

Confidence Level:

• This is a percentage telling how accurate your test is. It symbolizes how likely your entire population will behave as your sample population behaved. Most tests are done until they achieve a 95% Confidence Level, meaning you’re 95% certain that your sample population reflects the same behavior as would your entire population.

Confidence Interval:

• This is an interval representing the percentage difference from your sample results.  If your confidence interval is ‘+/-5’ and 60% of your sample population responded positively to your test, then 55% to 65% of your entire population will respond the same.

 Population:

• This is the number of total users in your entire population. For example, if your email list is 1MM people, then your ‘entire population’ would be 1MM users, your sample size may only need to be 5,000 to hit the confidence interval you’re looking for. You’ll derive the needed sample size from your population size and desired confidence interval.

Now that you understand the fundamentals of your needed sample size, prior to launching your test you should have an agreed upon confidence interval. Typically, a 95% confidence interval is an acceptable margin of error. Depending on the value of your test, you may be inclined to accept a lower confidence interval or if you’re testing a multi-million dollar outcome, you may want to see a 99% confidence interval.

Lastly, to ensure your sample users best represent your entire population, make sure they are randomized. Again, this oversight will skew results and not reflect the opinions or behaviors of your entire population. Here is a free confidence interval calculator to determine the sample size for a corresponding confidence interval and population size.

You’re now set, ready and excited to kick off your next creative asset test. For additional information on testing creative assets, refer to Testing Creative Assets for Stellar Performance. For all of your affiliate marketing needs, there is HasOffers; the leader in Affiliate Tracking Software. Please feel free to contact me or leave comments below

Remember the time your design team created that new landing page, call-to-action button, or email? You remember the one, the one that was going to be the end-all-be-all as the single greatest success in your creative career. Well, how’d it go? Was it as amazing as you thought it would be? If it’s like anything I’ve put out, it had a few more versions to go. The key to successful creative assets is to always be testing. If you’re running an affiliate program, the right affiliate tracking software will assist in testing, tracking and optimizations.

Before we jump into commonly tested components, I want to bring up establishing a hypothesis prior to launch. I’ve encountered many instances where people roll out tests and didn’t know what they were looking to observe. Make sure that you have an accurate hypothesis based on what variables or KPIs you’re testing and determine what each outcome will mean and represent. If you’re real good, you’ll have a post-test action plan.

Let’s move on to the many different components we can test. Please keep in mind; these are not all of them, just commonly tested ones that can produce medium to large improvements.

Copy

Whether it’s a title, headline, subject line or some other form of first-see-copy, make sure to provide; timely, targeted, relevant and appropriate copy. Use the 2 second rule; your headline has 2 seconds to grab attention and confirm users have found what they’re searching for. Alternatively, this can help deter users who aren’t looking for what you’re offering, saving you money if you’re paying per click.

Make sure to test different copy and appeal to your user’s emotions. You know the reasons your customers need your products or services, appeal to that emotion and reaffirm your solution. This reminds me, write copy that is customer-focused and not company-focused. This will indirectly increase your sales as you’re solving a problem rather than advertising a product.

Constantly test different variations of copy that appeals to your users, focus on; tone, style and theme. Look into the different demographics of your audience and see what resonates best with them.

Call-to-Action

The call-to-action encourages your user to do something, which makes it one of the most important assets to test and optimize. I recommend testing five to ten different variations of your call-to-action and test which renders the highest click-through rate. Remember to display and rotate your assets evenly. If you’re not randomizing, your results will be skewed.

Common call-to-action text include; “Start”, “Submit”, “Go Now” or even the age old “Click Here”. Internet users have become numb to these phrases over the years. Still test them, but also test longer messaging that provides more targeted information. For example, a retailer might want to A/B test the following text; “Buy Now” vs. “Proceed to Checkout”. More information in your call-to-action button will do two things; increase your click-through rate and better qualify users.

Test the following when optimizing your call-to-action buttons; copy, length, color and size. Test messaging that is short and sweet versus informative and lengthy. Try as many different colors as possible, when you find a winner, test different shades and gradients. Test the size of the call-to-action, studies have shown size relative to its surroundings can make incremental improvements to click-through rates. Which leads to my last point, test the various locations on your site where your call-to-action is placed.

Landing Page / Sign-Up Page

The length of your form will largely dictate the success of your conversion rate. A short form (typically 4 to 8 fields) can yield much higher conversion rates than long forms (8+ fields). I would start by, what information is critical to capture? A lot of companies use required fields and optional fields. This is a great idea, make sure to A/B test required + optional vs. just required. You may decrease your conversion rate by displaying too many fields (required + optional) to your users and overwhelm them.

Adding credibility logos to your site can increase user confidence and provide a sense of security. I’ve seen mixed results with credibility logos, so plan to test several of these. In addition to increasing user confidence through credibility logos, experiment with testimonials at the bottom on your conversion page. Users feel more confident knowing others have had success when interacting with your site. Use your Thank You emails to incentivize users to post testimonials on your site.

I’d also recommend displaying inbound phone numbers on your purchase or conversion pages. These appeal to users who don’t feel comfortable transmitting personal information over the internet. Additionally, less technologically savvy users convert or purchase over the phone more than online. Utilize a third party call tracking systems to correlate your conversions to specific media channels or affiliates. RingRevenue is incorporated in your HasOffers affiliate software to track inbound calls.

Results

No one is perfect and therefore no asset will be either. Be patient and think creatively, you’re going to find greater successes when you think outside the box. At first, you’ll see big wins and you’re going to need to harness that energy when it comes to squeezing the lemon for those 10 basis point increases in the future. Don’t spend too much time on the little victories, start thinking about your next big success and move quickly.