Test a widget change against your live version before publishing it

Every engagement widget's editor gets a seventh tab: Experiment. It lets you run your draft against your currently live version with real traffic split between them, instead of guessing from a before/after comparison.

Start a test from a widget with a published version and a draft, pick how much traffic goes to the draft (10 to 50%), and the two run side by side until there's enough data to call a winner. The test measures conversation start rate and uses an "always-valid" statistical method, so you can check in on it anytime without that habit skewing the result. The trade-off is that it needs a real amount of traffic: no verdict renders until each version has been assigned at least 5,000 sessions. Below that, you still see both versions' raw numbers, just no verdict.

A few things worth knowing going in: traffic is split by browser session, not by shopper, so someone browsing in two tabs can land in both versions. Both versions have to target the same pages: this release can test a widget's look, copy, size and placement, not which pages it appears on. And "not yet conclusive" never means "no difference". It means the sample size can't yet tell a real difference from noise; the tab explains the detection limit rather than implying a tie. If the candidate is clearly hurting the experience, the system reverts all traffic to your live version automatically.

See A/B testing a widget for the full picture, including how a test ends.