AB test summary

 目录

一、Common business questions：

二、AB测试步骤：

三、影响ABFactors affecting the accuracy of test results

四、AB测试效果分析 

 1.Experiment validity analysis

 2.实验结果的分析

---

A/BTesting is the most practical way to drive continued business growth,最有效的方式

 一、Common business questions：

• 产品迭代：Change the user interface to improve the user experience,Optimize the registration process for new users to increase conversion rates,Determine the most valuable product coupons,Add product features to improve user retention
• 算法优化：Improve user stickiness by improving the accuracy of recommender system algorithms,Improve the click-through rate of results by improving the accuracy of the search ranking algorithm,Improve the click-through rate of your ads by improving the accuracy of the ad display algorithm
• 市场营销：Determine the optimal marketing content,Determine the optimal marketing time,Identify the most precise audience,Measure the effectiveness of marketing

二、AB测试步骤：

  根据试验结果确定发布新版本、调整分流比例继续测试或者在试验效果未达成的情况下继续优化迭代方案重新开发上线试验.

三、影响ABFactors affecting the accuracy of test results

1.样本数量：Flow sample determination

2.样本质量：Whether the split sample is valid

 3.The length of the test

 4.Interaction of multiple experiments in parallel

（For details, see the articles recommended by other bloggers at the bottom of the article）

四、AB测试效果分析 

关于ABThe analysis of experimental effects is usually divided into two steps：Judgment of the validity of the experiment、实验结果的比较.

 1.Experiment validity analysis

①Determine whether the triage of the experiment has reached the required minimum sample size,Thus, the occurrence of two types of statistical errors can be rejected with a high probability.The judgment of the minimum sample size can be made under the assumption that the experimental target index conforms to a normal distribution,The quantiles of the probability of occurrence of the two types of errors are estimated;

②To judge the validity of the sample.采用AA测试,如果AAThere were no significant differences in the results of the experiments,Then the experimental results can be considered valid,Further judgment can be made on the experimental results of the old and new versions;

③Determine whether the test time meets the sample requirements,It also takes into account adaptation periods and behavioral cycles;

④Determine whether it is affected by parallel experiments.

 2.实验结果的分析

After confirming the validity of the experiment, the results of the experiment can be judged,Usually by comparing whether there are significant differences between the new experimental version and the old version（前述的P值判断）,As well as calculating the confidence interval for the index of the experimental results（indicators are usually selected95%置信区间）,Thereby, it can be judged whether the new version has a significant improvement or decline relative to the old version.
 

Here are other bloggers who have organized them in detail and I think it is very good to push them to everyone. 

 (24条消息) 【数据分析】A/B测试_Fushi's Blog-CSDN博客_aa测试

 (24条消息) 浅谈A/B测试 ,看这一篇就足够了_佞臣888的博客-CSDN博客_a/b测试

 (24条消息) A/B测试_sd116460的博客-CSDN博客