Embedded Files
How to perform a statistically significant A|B test
How to perform a statistically significant A|B test
Exploratory data analysis
Understand Customer Page path and Map User Journey
Data dimensional modelling (split)
Define Null (Control/A) and Alternate (Challenger/B) Hypothesis
Choose a statistical test
Perform a one-sided, paired, t-test
Summary Statistics
Develop visualisation of data (raw and standardised)
Interpret test result
Data Distribution of Customer Behaviour on Website
Data Distribution of Customer Behaviour on Website
By designing a dimensional data model and utilising statistics, we can see the distribution of customer-journeys as each customer attempt to complete their applications on the website.
Data distribution is split into two groups;
'before content changes were made to the website' and 'after content changes were made to the website'
Central Limit Theorem: As long as the sample size is significantly large (n ≧ 30), the distribution of the mean will be approximately normal.
Central Limit Theorem: As long as the sample size is significantly large (n ≧ 30), the distribution of the mean will be approximately normal.
Choosing Your Test
Choosing Your Test
Performing contextual analysis on the problem statement led to choosing a statistical test with the following characteristics;
Performing contextual analysis on the problem statement led to choosing a statistical test with the following characteristics;
Numerical data type
Sample size is (according to central limit theorem) assumed to be normally distributed
Samples are paired
Direction os test is two-sided
The only statistical test that meets all criterias above anmd effectively solves the problem statement is a; Numerial, paired, two-sided, t-test.
The only statistical test that meets all criterias above anmd effectively solves the problem statement is a; Numerial, paired, two-sided, t-test.
Standardise the data and develop histogram showing distribution
Standardise the data and develop histogram showing distribution
Test Results and Summary Statistics
Test Results and Summary Statistics
Test statistic = -0.2070
P-value = 0.8367
Significance level (𝜶) = 0.05 (5%)
Interpretation and Recommendations
Interpretation and Recommendations
We cannot conclude that changing the content on the “Before You Start Page” affected the proportion of users completing an application first time.
The test result is not statistically significant enough to reject the null hypothesis.
This does not mean we have proved the null hypothesis.
Page updated
Google Sites
Report abuse