PHASE II: EXECUTING THE EXPERIMENT
Step 5 – Produce:
The preparation phase should already determine what to create and how to create it. In production, the actual design or copywriting work may begin.
Step 6 – Test:
Real execution may begin as soon as test variants are produced. Execute the experiment by:
- Picking the right testing method.
- Choosing a suitable testing platform.
- Deploying variants to the platform, following conditions of the chosen method.
- Beginning the test.
Step 7 – Measure:
Compare the Control variant’s conversion rate (CVR) with the test variant’s to measure test results, then validate or invalidate the hypothesis, and finally make sense of the results – with critical reflection, not assumptions.
Step 8 – Handle the results:
There are 3 standard protocols to follow for the treatments of the results at this stage: Scale the variant/test, scrap it, or adjust it.
- Scale a convincing win by expanding the variant to more languages, assets, etc., while keeping the same strategy. Scaling starts from production – making a growth loop for CRO.
- Scrap a loss that shows little to no value and is beyond salvation. Either revisit a previous step in the Preparation phase or restart the whole process and do the research again. This is a feedback loop that at least guarantees to learn in CRO.
- Adjust anything in between, as long as it is possible and worthwhile to improve the variant. Change one detail at a time, develop a new hypothesis, then test again, and repeat. This is where the true optimization loop is made – with iterative testing.
If you would like to learn more about our frameworks or need help implementing it, don’t hesitate to contact us at [email protected].
See you around!