Shopify now has built-in A/B testing. As of June 5, 2026, you can run a Shopify A/B test on a whole theme or your checkout from inside your admin, through a feature called Rollouts. No third-party tool needed.
A/B testing on Shopify itself isn't new. Merchants have split-tested headlines, images, and checkout flows for years with apps, page builders, and tools like Optimizely or VWO. What's new is running that test natively at the theme and checkout level, the parts an external tool could never touch cleanly.
1. What's new: native A/B testing in Rollouts
On June 5, 2026, Shopify expanded a feature called Rollouts to support scheduling, gradual publishing, and A/B testing for two things: your online store theme, and your checkout and customer account configurations. You'll find it under Markets > Rollouts in your admin.
A rollout is a scheduled set of changes to your main theme, your checkout and account pages, or both. Think of it as a controlled way to push changes live instead of flipping a switch and hoping for the best.
Here's what that lets you do now:
- Schedule a go-live. Set a new theme or checkout to publish at a specific date and time, like switching from Dawn to Horizon the morning a campaign starts.
- Swap temporarily, then auto-revert. Run a different theme or checkout for a fixed window, such as a BFCM setup for one week, and let it roll back on its own.
- Roll out gradually. Release a new theme or checkout config to a percentage of visitors first, like a redesigned customer account page, before you go wider.
- Run an A/B test. Put two entirely different themes or checkout setups head-to-head to find the winner, such as testing a new upsell block on your checkout page.
- Test localized content per market. Schedule or test different content by region, like different CTA copy per market, to see what converts where.
A couple of details make this safer to run on a live store. When you start a rollout, Shopify automatically copies your published theme or config, so you can keep editing your live store while a test runs. And because experiments can be mutually exclusive, you can run more than one at the same time without them overlapping on the same visitors.
The native testing works at two levels: a whole theme (Dawn vs Horizon, say) or your checkout and account configuration. It compares entire experiences against each other, not single elements.
1.1. Launch vs experiment
A rollout runs as one of two types: a launch or an experiment. Only the experiment is an A/B test.
| Launch | Experiment (A/B test) | |
|---|---|---|
| What it does | Shows one version to visitors | Splits visitors between two versions |
| Traffic | No version split | Control vs treatment, side by side |
| Best for | Scheduled or gradual rollouts | Comparing two options to find a winner |
| Plan needed | Basic or higher | Grow or higher |
A launch applies your changes to all eligible visitors, with no split. Good for a seasonal theme or a scheduled sale.
An experiment splits eligible visitors between a control (your current experience) and a treatment (the new version), so you compare them on live traffic without betting the whole store on the change.
A quick note on scope, since A/B testing itself isn't new here. Most tests merchants run are element-level: a headline, a button, one offer, or which product recommendations show in the cart. Those still run through apps or a page builder. What Rollouts adds is the missing piece, native testing for the whole theme or checkout.
2. Plans, permissions, and requirements
Native A/B testing isn't on every plan. Here's who can use what.
Plan requirements:
- Rollouts (scheduling and gradual launches) are available on the Basic plan or higher.
- Experiments, the A/B testing part, need the Grow plan or higher.
So scheduling and gradual rollouts are open to almost everyone. The head-to-head A/B test is gated to Grow and up. Element-level testing through apps still works on any plan, so a Basic-plan store isn't shut out of testing entirely.
Who can set it up: the store or organization owner, anyone with an Administrator role, or a staff member whose store role includes the Markets > Rollouts permission. To add specific changes, they also need the matching permission: "View and edit checkout and customer accounts" for checkout configs, and Online store > Themes for theme changes.
Quick gut-check: not sure which plan you're on? Check Settings > Plan in your admin. Below Grow, you can still schedule and gradually launch changes. You just can't run a split experiment yet.
3. How to set up your first Shopify A/B test in Rollouts
3.1. Before you start
Build the thing you want to test first.
- Testing a theme replacement? Add the new theme under Online Store > Themes before you create the rollout.
- Testing a checkout change? Create the new checkout and accounts configuration first.
- Editing your current main theme? You'll save the rollout first, then edit it.
3.2. Step 1: Create and name your rollout
Go to Markets > Rollouts and click Create rollout. Give it a name you'll recognize later, like Horizon theme test or Checkout upsell test.
Access the Rollouts through Shopify Admin > Markets
3.3. Step 2: Add your changes
Click Add changes to your store, then pick what you're testing:
- Online store theme: edit your main theme in the editor, or choose "replace your main theme with another" and select the new one.
- Checkouts and accounts: click "replace your current configuration with another" and select the config.
Create changes to your store
You can add one theme change and one checkout change per rollout.
3.4. Step 3: Edit rollout details and set the rules, schedule time
This is the step that makes it an A/B test rather than a plain launch.
First, set your launch reach: the percentage of total visitors the rollout touches at all. Then, in the Change section, click where it says "100% of eligible visitors will see this" and drop that percentage down. The moment you serve the change to a subset instead of everyone, Shopify treats it as an experiment and starts comparing your treatment against the control.
Configure details, the launch rules, and schedule time for your rollouts.
A worked example: set launch reach to 50% and the change to 50%, and your treatment shows to 50% of that eligible half. The rest act as your control.
One caution worth repeating. If you launch a rollout immediately at 100%, Shopify collects no analytics, and you can't revert it. A real A/B test needs a start and end date plus a sub-100% split to measure anything.
3.5. Reading your results
Once the experiment is live, click it on the Rollouts page to see analytics. The metrics depend on what you're testing.
| What you're testing | Key metrics you'll see |
|---|---|
| Theme experiment | Conversion rate, bounce rate, reached-checkout rate, added-to-cart rate |
| Checkout experiment | Checkout conversion rate (completed purchases relative to sessions that reached checkout) |
Test both at once and the metrics show up in separate Storefront and Checkout sections. For the full breakdown, Shopify's own Rollouts documentation lists every available metric.
4. A/B testing best practices that actually matter
A native testing tool won't rescue a badly run test. A handful of rules decide whether your numbers mean anything.
- Test one change at a time. Swap the theme, the checkout, and the offer all at once, and a lift tells you something worked, but not what. Isolate the variable so you can act on the result.
- Wait for enough traffic. Small samples lie. A 6% versus 5% result across a couple hundred visitors is noise, not a winner. CRO practitioners usually aim for a 95% confidence level before calling it. If your store doesn't pull steady traffic, whole-page testing may not be worth it yet. Talk to customers or watch session replays instead.
- Run it for full weeks, not days. Shopping behavior swings between weekdays and weekends. Run a test across at least two full business cycles, usually two to four weeks, so one good Monday or slow Sunday doesn't skew the call.
- Don't stop early. An early "winner" often drifts back to the mean. Let the test reach its planned end date or sample size before you decide anything.
- Start from a hypothesis. "Test the checkout" isn't a plan. "Removing the forced account-creation step will lift checkout completion, because mandatory sign-up adds friction" is. A clear hypothesis makes even a losing test useful, because you learn something either way.
Learn more: How guest checkout cuts friction on Shopify
5. What's worth testing, and what you still need apps for
Native Rollouts handles the big swaps: a whole theme, a whole checkout config. But plenty of high-impact tests are smaller and more specific. Common targets:
- Headlines and CTAs: copy, color, and placement.
- Product images: studio shots vs lifestyle, static vs video.
- Pricing presentation: how discounts, comparisons, and bundles appear.
- Social proof: where reviews and ratings sit on the page.
- Cart and upsell offers: what you suggest, and when.
- Navigation menus: how fast shoppers find what they want.
Most of these live inside a theme or checkout flow. Rollouts splits the traffic. You still build the two versions you're comparing, and for element-level changes that means an app or a page builder.
You can bring in third-party apps at this stage. If you're testing checkout configurations, the differences you're comparing are the customizations themselves. For Shopify Plus merchants, that usually means building a branded, conversion-focused checkout as your treatment, then letting Rollouts tell you whether it beats your current setup.
If you're testing theme or cart changes, the upsell offers, quantity-break bundles, and cart-drawer experience you're comparing are all built with apps. Set up two versions of your cart, say one with a free-gift offer and one without, let Rollouts split the traffic, and keep the one that wins.
The pattern is simple: Shopify decides the winner, your apps create the contenders.
6. Start testing
Whether you've A/B tested for years or you're running your first one, the native tooling lowers the barrier. If you're on Grow or above, open Markets > Rollouts and run one small test this week. Pick one thing you've been unsure about, set a 50/50 split, write a one-line hypothesis, and let real shoppers settle the debate.
The checkout is often the highest-stakes thing to test, and the hardest to build a strong variant for. If that's where you want to start, Qikify's checkout specialists will review your checkout page and map out the changes worth testing → Book a checkout consultation.
