A\/B testing, evaluating two variants against each other, is the most effective way to increase the usefulness and profitability of digital applications with little effort.<\/p>\n\n\n\n
That\u2019s why, here at SMARTASSISTANT, we\u2019re big on A\/B testing. It allows for immediate feedback from your target audience on how well you are advising and guiding shoppers to the right products.<\/p>\n\n\n\n
Much like any other sales associate, a digital advisor requires continuous training so that it can give the best advice and keep up with an ever-changing audience. A\/B testing is this training.<\/p>\n\n\n\n
In this article, I\u2019ll share the biggest A\/B testing mistakes to avoid.<\/p>\n\n\n\n
First off, the biggest mistake you can make is simply not A\/B testing.<\/strong><\/p>\n\n\n\n Many companies have great excuses as to why they\u2019re not A\/B testing or simply don\u2019t see the need for it. Their solution is live, customers are using it, project done.<\/p>\n\n\n\n Not quite.<\/p>\n\n\n\n Customer habits and expectations are changing rapidly today, as are products and technologies. Whenever you are in the business of serving people, you can\u2019t afford to stand still when everything around you is changing. With A\/B testing, you are able to detect these subtle changes, make adjustments, and evolve accordingly.<\/p>\n\n\n\n In my time as Conversion rate optimization consultant, I have seen that simple but smart adaptations based on A\/B testing results can increase the conversion rate by over 9% or increase revenue by 15%.<\/p>\n\n\n\n Neglecting the need to A\/B test means leaving money on the table in almost every scenario.<\/p>\n\n\n\n However, there are a few acceptable excuses as to why you cannot yet A\/B test:<\/p>\n\n\n\n These are the only situations that should hold you back. Once you\u2019ve spent some time addressing them, you can then begin confidently A\/B testing.<\/p>\n\n\n\n Here\u2019s how to go about starting A\/B testing:<\/strong><\/p>\n\n\n\n No detail is too small for A\/B testing, but there is such a thing as putting the cart before the horse. Addressing small conversion details before the potential glaring issues would be doing just that.<\/strong><\/p>\n\n\n\n The color of your buttons, for example, isn\u2019t important enough to test first.<\/p>\n\n\n\n The wording of your question and answers, integrated images, the position of buttons, and the fluidity of the experience are all far more notable elements to be focused on.<\/p>\n\n\n\n A\/B testing them can yield some interesting insights and impressive results.<\/p>\n\n\n\n Here\u2019s how to avoid this mistake:<\/strong><\/p>\n\n\n\n When not handled properly, changing multiple elements at once can completely nullify your A\/B testing efforts.<\/strong><\/p>\n\n\n\n This is because you won\u2019t be able to attribute your test results to any particular change.<\/p>\n\n\n\n For example, if you change the first question in your digital advisor and add new images at the same time, you won\u2019t know which adaptation led to changes in user behavior.<\/p>\n\n\n\n Without clarity, you can\u2019t proceed with the data you\u2019ve gathered.<\/p>\n\n\n\n\n
<\/strong> Without this base understanding, you won\u2019t know what the metrics you are looking at mean.<\/li>\n\n\n\n
<\/strong> At the end of the day, if you don\u2019t know what you\u2019re looking at, you can\u2019t do anything meaningful with it.<\/li>\n\n\n\n
<\/strong> If you\u2019re making a handful of sales a month, data swings are common. It makes testing irrelevant, as you won\u2019t be able to extrapolate anything from it.<\/li>\n<\/ol>\n\n\n\n\n
<\/strong> Questions, Answers, CTA, Images, Q&A flow etc.
Tip:<\/strong> The wording of the questions can play a huge role in how customers respond to your advisors.<\/li>\n\n\n\n
<\/strong> Blindly taking a stab at changing something<\/em> will not yield meaningful results. Hypotheses define your assumptions and will help you focus.
Tip:<\/strong> You will always get the best data by only having one hypothesis per variation.<\/li>\n\n\n\n
<\/strong> Without knowing your goal, you\u2019ll never know when you reach it. Testing just \u201cto see what the impact is\u201d is not the way to go.
Tip:<\/strong> When evaluating digital advisors, useful metrics to consider are the conversion rate,<\/em>\u00a0\u00a0click out success rate<\/em> (share of sessions that contain a click out), bounce rate<\/em> (share of sessions that bounced after opening the advisor) and click-through success rate<\/em> (share of sessions that contain a final page). It is also very important to choose only one key metric to improve per A\/B test.<\/li>\n<\/ol>\n\n\n\n2. Focusing On Small Details First<\/h2>\n\n\n\n
\n
<\/strong> As you test, you\u2019ll realize some elements are more or less important to your audience than others. Make notes of these!<\/li>\n\n\n\n
<\/strong> Ensure your sample size is large enough to draw conclusions before you launch the test and before implementing any changes.<\/li>\n\n\n\n3. Changing Multiple Variables at Once<\/h2>\n\n\n\n