Split Testing Through Campaign Evolution

In our last post we covered why A/B testing can be difficult for some companies to effectively implement. But that doesn’t mean the principles of testing should be abandoned completely.  An evolutionary process of consistent improvement is a more gradual way of implementing split tests.

Many trainers, consultants, and professional coaches set up a template for a marketing campaign, run it for a period of time until they get sick of it, and then do a redesign which starts the process over again.  While this keeps them up to date on new trends in marketing and technology, it’s not introducing improvements as the campaign runs like A/B testing will.

A/B testing at its best is a duplicate communication with one specific difference.  That difference can then be tested for effectiveness and the better performing treatment is then adopted. Digital marketing campaigns should have some level of repetitiveness especially in layout and design.  These repeating elements can be leveraged as a control and updated one at a time and compared for effectiveness over time in the same way that A/B tests are.

Making gradual split tests while running a digital marketing campaign avoids the common limiting factors of A/B testing but still allows for ongoing testing for gradual improvement.  However, there are a few restrictions to keep in mind.

Time

Time is the primary limiting factor in doing gradual split tests. Because the sends are more spread out, changes cannot be implemented as quickly.  Make sure you allow enough time on a single change to gather sufficient information.  For example, if you have a monthly newsletter you’ll need to run the change twice to validate a changes effectiveness which means each change will take three months to validate.

One Change at a Time

This is really another limiting factor of time but subtly different.  Split testing relies on testing a single element to know that particular change is responsible for an improvement or decline. Having more than one thing changed to speed up the process only serves to invalidate your test.

Same Audience

Since there is a gap of time between treatments you need to keep consistency with the audience. Too many changes into who receives the communication will serve to invalidate the test.

Content

While many elements are repetitive in digital marketing, content often is not.  If you have small elements of recurring content, like an email subject line with repeating title or commonly used social media tags, then by all means test it.  But most of the content variables will not repeat consistently enough to be tested in a gradual ongoing method.

If you plan for these restrictions and formulate gradual split test changes around them, you can gather many of the same insights that A/B tests will provide without dedicating nearly as much time or as many resources.