The Experiment in Action
Hypothesis
Shipping is a key differentiator for many eCommerce shops. Most shops offer multiple shipping options to meet the increasing demands of customers.
That being said, having to select a shipping option adds an additional step in the checkout process.
Since many eCommerce shoppers will select the most economical shipping method, auto selecting the most economical method could increase conversion rates and revenue per visitor (RPV).
CONTROL – MOBILE
TREATMENT – MOBILE
Auto Select Shipping Method Experiment
Results: Auto Select Shipping Method Experiment
Average performance numbers are based on results from November 7, 2018 to September 15, 2019 from 7 merchant sites.
Average Experiment Results So Far*
| Device Type | Average Lift | Visitors | |||||
| Revenue per Visit (RPV) | Conversion Rate (CR) | Average Order Value (AOV) | Treatment | Control | |||
| +2.08% | -0.22% | +2.32% | 23,817 | 21,552 | ||
| -0.08% | +0.60% | -0.26% | 15,183 | 13,091 | ||
| -1.24% | +1.66% | -3.00% | 3,711 | 3,253 | ||
Results: Auto Select Shipping Method Experiment
Average performance numbers are based on results from November 7, 2018 to September 15, 2019 from 7 merchant sites.
Average Experiment Results So Far*
 | Mobile |
| Average Lift | ||
| Revenue per Visit (RPV) | Conversion Rate (CR) | Average Order Value (AOV) |
| +2.08% | -0.22% | +2.32% |
Total Visitors: Treatment: 23,817 | Control: 21,552
 | Desktop |
| Average Lift | ||
| Revenue per Visit (RPV) | Conversion Rate (CR) | Average Order Value (AOV) |
| -0.08% | +0.60% | -0.26% |
Total Visitors: Treatment: 15,183 | Control: 13,091
 | Tablet |
| Average Lift | ||
| Revenue per Visit (RPV) | Conversion Rate (CR) | Average Order Value (AOV) |
| -1.24% | +1.66% | -3.00% |
Total Visitors: Treatment: 3,711 | Control: 3,253
Distribution of Average RPV Lift
for All Visitors
The RPV lift distribution suggests that even a subtle test like this one can provide modest lifts to RPV for the average merchant.
Distribution of Average Conversion
Rate for All Visitors
This is confirmed by the conversion rate distribution where 90.8% of merchants saw a modest lift (up to +2%) in conversion rate.
*These results reflect the average findings for all participating merchants and are not guaranteed for all users.
Data Analysis: Auto Select Shipping Method Experiment
Things to Consider
This test leverages the specific shipping management platform, ShipperHQ, so the sample size is still small at this time. As more data accumulates, we should gain more insights into user behavior related to selecting shipping options.
Who Should Try It
All merchants with multiple shipping options should consider running this test. Despite being a subtle change, merchants may find at least one device type — likely desktop — that will benefit from the auto selection of the cheapest shipping method.
Who Should Try It
In isolation, the floating add-to-cart button may not have an immediate positive impact. However, testing the floating add-to-cart button in conjunction with optimized post add-to-cart transitions can have greater impact on RPV.
Merchants that already optimized add-to-cart transition behaviors are likely to succeed with the floating add-to-cart button experiment. Merchants that utilize a standalone pop up modal should still consider this experiment, since every shop is different and other unknown factors may still help achieve to success.
Interested in Learning More?
Get deeper insights into over 130 experiments available through the Initiative.