Experiments

AUTO SELECT SHIPPING METHOD

Auto Select Shipping Method

The Experiment in Action

Hypothesis

Shipping is a key differentiator for many eCommerce shops. Most shops offer multiple shipping options to meet the increasing demands of customers.

That being said, having to select a shipping option adds an additional step in the checkout process.

Since many eCommerce shoppers will select the most economical shipping method, auto selecting the most economical method could increase conversion rates and revenue per visitor (RPV).

CONTROL – MOBILE

Auto Select Shipping Method - Control

TREATMENT – MOBILE

Auto Select Shipping Method - Treatment

Auto Select Shipping Method Experiment

How It Works

How It Works

For merchants that offer multiple shipping options (with none pre-selected), this treatment auto selects the cheapest shipping option for the user. The user can change their selection at any time.
null

Why We Are Testing It

Amazon and Walmart have increased fulfillment competition in the eCommerce industry. As a result, shipping is a hot issue for many merchants.

This experiment aims to optimize merchants’ shipping offerings — simplifying a step in the critical checkout process — and increasing the likelihood of a conversion.

Results: Auto Select Shipping Method Experiment

Average performance numbers are based on results from November 7, 2018 to September 15, 2019 from 7 merchant sites.

Average Experiment Results So Far*

Device TypeAverage LiftVisitors
Revenue per Visit
(RPV)
Conversion Rate
(CR)
Average Order Value
(AOV)
TreatmentControl
Device Type: MobileMobile
+2.08%-0.22%+2.32%23,81721,552
Device Type: DesktopDesktop
-0.08%+0.60%-0.26%15,18313,091
Device Type: TabletTablet
-1.24%+1.66%-3.00%3,7113,253

Results: Auto Select Shipping Method Experiment

Average performance numbers are based on results from November 7, 2018 to September 15, 2019 from 7 merchant sites.

Average Experiment Results So Far*

Device Type: MobileMobile
Average Lift
Revenue per Visit
(RPV)
Conversion Rate
(CR)
Average Order Value
(AOV)
+2.08%-0.22%+2.32%

Total Visitors: Treatment: 23,817 | Control: 21,552

Device Type: DesktopDesktop
Average Lift
Revenue per Visit
(RPV)
Conversion Rate
(CR)
Average Order Value
(AOV)
-0.08%+0.60%-0.26%

Total Visitors: Treatment: 15,183 | Control: 13,091

Device Type: TabletTablet
Average Lift
Revenue per Visit
(RPV)
Conversion Rate
(CR)
Average Order Value
(AOV)
-1.24%+1.66%-3.00%

Total Visitors: Treatment: 3,711 | Control: 3,253

Distribution of Average RPV Lift
for All Visitors

Distribution of Average RPV Lift - Auto Select Shipping Method RPV

The RPV lift distribution suggests that even a subtle test like this one can provide modest lifts to RPV for the average merchant.

Distribution of Average Conversion
Rate for All Visitors

Distribution of Average RPV Lift - Auto Select Shipping Method CR

This is confirmed by the conversion rate distribution where 90.8% of merchants saw a modest lift (up to +2%) in conversion rate.

*These results reflect the average findings for all participating merchants and are not guaranteed for all users.

Data Analysis: Auto Select Shipping Method Experiment

What It Means

What It Means

When visitors are close to completing their purchase, configuration changes may be less likely to impact the average order value (AOV). Since this experiment takes place at the buying stage of the checkout funnel, the conversion rate is an important metric to consider.

The findings for this experiment indicate that mobile visitors may be more inclined to switch to a more expedited shipping option – or at least take more time to consider the options.

Things to Consider

This test leverages the specific shipping management platform, ShipperHQ, so the sample size is still small at this time. As more data accumulates, we should gain more insights into user behavior related to selecting shipping options.

Who Should Try It

All merchants with multiple shipping options should consider running this test. Despite being a subtle change, merchants may find at least one device type — likely desktop — that will benefit from the auto selection of the cheapest shipping method.

Who Should Try It

In isolation, the floating add-to-cart button may not have an immediate positive impact. However, testing the floating add-to-cart button in conjunction with optimized post add-to-cart transitions can have greater impact on RPV.

Merchants that already optimized add-to-cart transition behaviors are likely to succeed with the floating add-to-cart button experiment. Merchants that utilize a standalone pop up modal should still consider this experiment, since every shop is different and other unknown factors may still help achieve to success.

Interested in Learning More?

Get deeper insights into over 130 experiments available through the Initiative.