Choosing the right Optimisation Metric Goals for A/B Test
Experimentation.Analytics.Customer behaviour research. A/B testing. These elements strengthen growth marketing to differentiate itself from another form of marketing.
"Our success at Amazon is a function of how many experiment we do per year; per month, per week, per day." Jeff Bezos
This week, I will be talking about information gathering and analysis.
As it is today, no organization or business relies much on instinct in decision making, even UX designer that usually works on instinct, still employs empathy mapping and research to design, based on usability principle. So, no serious business decides customer behaviour evaluation without sufficient data.
It is all about data.
That is a good, of course, approach to business growth and there is a lot of ways to gather information without having to solely depend on a source. And the commonly data-driven approach used is A/B testing.
To make a good sense of A/B testing efficacy, I will reference Dan Siroker, the co-author of the book "A/B Testing" and Tom Wesseling of CXL Institute. Dan Siroker et al, expatiated on the adoption of more than 200 times to use A/B testing for fundraising for Barrack Obama presidential campaign in 2008, and huge success was recorded. They both concluded that information gathering is insightful into customer behaviour and capable of driving digitalized business growth.
What is A/B Testing?
In Tom Wesseling A/B Testing Mastery class, he alluded to A/B testing to evaluation of behaviour and to change behaviour, to create more transactions, more leads, more clicks, or whatever you offer. But in a basic way, A/B testing is an approach of comparing two versions of something to make a sense of the best-performed version.
A/B Testing is as old as human history. Only that it was not associated with website and phone apps as it is now but used in the 1770s by Dutch people on the big ships going to the east to differentiate infected people from non-infected people, and those who got what disease or not. However, the advent of websites in the 90s enthralled A/B testing.
A/B Testing has since that period defined itself. You cannot indiscriminately use it without knowing what you want to test or achieve. Your purpose or goal is the determinant in its deployment. That is why A/B Testing uses goal metrics instead of Key Performance Indicators (KPI).
Do you want to test for navigation leads, call to action, copy, and layout of your website? The purpose is what informed your evaluation decision. You need to know whether you want to evaluate the performance of any page or button on your website. For instance, if you are changing your call-to-action button color or overhauling the button tag to something different. So to run a test with this, you will show the original and variant to a segment of your users to test the one which converts the most before pushing it out to your wide customers or users. Is it the original or the variant?
Tom Wesseling emphasized Randomised Control Experiment in his A/B Testing Mastery. RCT allows the greatest reliability and validity of the statistical estimate of the treatment effect. For instance, in testing for navigation leads, the old and new design should be allocated to two segments of users randomly to reduce cognitive bias.
The allocation can be for mobile users and the desktop user is given the fact that mobile user experience about the new design might be different from the desktop user. Dividing the users between mobile and desktop and randomly assign the new design will help have utter insight and behavior of different users toward the new design.
How do you pre-design A/B Testing?
Many things go into pre-designing for A/B Testing. As I said, you can't indiscriminately start A/B Testing without knowing if you have to or not. The first thing, you have to do is to:
* Research: you need to know the element relevant and making an impact on your site and this is known as a conversion signal map-looking.
In recent time a digital marketing company interviewed me to seek my opinion about their website. I wanted a few digital user psychologies but I knew it would not be necessary as relevant elements that can make an impact on users' behaviour have not been determined. Besides the fact that the site lacks what should be expected of a B2B site, it'll make no difference without deep insight into relevant elements.
Customer behaviour study goal which will help to gain insight into the most important customer journey, understanding basic user behaviour and inputting for the setting hypothesis.
Most time, tools like hotjar and Google analytics are the first step to gain insightful exploration of the user or customer behaviour and their journey, then the voice of customer research.
The reason voice of customer research is equally important as analytics only show behaviour but does not say why.
After gathering information, having enough data, it is easy to know what to test. So, you can develop a hypothesis based on the psychological-driven and data-driven statement. For instance, based on research and data collated, "If I apply this then this behaviour will happen among this group because of their reaction or behaviour."
A/B Testing is not an alpha and omega of it all, there are some other approaches and methodologies that can equally produce better results. However, it is all hinged on purpose and goal. Sometimes, UX research can produce a great result if you have no enough data to run because you need a monthly 1000 visitors before you can run A/B Testing.
