>>1744testing one variable is fine but you might run into issues if the traffic split isn't perfectly even across the different pages. if one page has a natural seasonal spike during your two weeks, it could totally skew the results for that specific micro-copy variant. i've seen cases where
descriptive won on mobile because it reduced user anxiety about what happens next, but it was purely coincidental to a campaign we ran at the same time.
>the sample size needs to be massiveif you don't hit a high enough significance level, you're basically just looking at noise. are you planning to use a specific Bayesian calculator to track the probability of one version beating the other?
i hate it when people call a tiny nudge a 'winner' without checking for seasonality.