Experiments

HIDE CHAT IN CART AND CHECKOUT

Hide Chat in Cart and Checktout

The Experiment in Action

Hypothesis

As we’ve seen from the results of experiments like Remove Wishlist and Coupon Collapse, additional elements on a cart or checkout page can distract users and prevent them from completing their purchase.

As such, suppressing chat elements on cart and checkout pages could reduce distractions and encourage visitors to complete their purchases faster.

CONTROL – MOBILE

Hide Chat in Cart and Checktout - Control

TREATMENT – MOBILE

Hide Chat in Cart and Checktout - Treatment

Hide Chat in Cart and Checkout Experiment

How It Works

How It Works

This experiment automatically detects chat elements on cart and checkout pages and hides them to reduce distractions for shoppers.
null

Why We Are Testing It

Distracting elements may pull visitors away from their intention to check out. Eliminating the chat function from the cart and checkout pages could help to keep the focus on completing the checkout process.

Results: Hide Chat in Cart and Checkout Experiment

Average performance numbers are based on results from March 06, 2019 to September 15, 2019 from 7 merchant sites.

Average Experiment Results So Far*

Device TypeAverage LiftVisitors
Revenue per Visit
(RPV)
Conversion Rate
(CR)
Average Order Value
(AOV)
TreatmentControl
Device Type: MobileMobile
+15.22%+11.47%+3.72%20,69321,219
Device Type: DesktopDesktop
+1.71%-1.19%+2.82%22,40620,436

Results: Hide Chat in Cart and Checkout Experiment

Average performance numbers are based on results from March 06, 2019 to September 15, 2019 from 7 merchant sites.

Average Experiment Results So Far*

Device Type: MobileMobile
Average Lift
Revenue per Visit
(RPV)
Conversion Rate
(CR)
Average Order Value
(AOV)
+15.22%+11.47%+3.72%

Total Visitors: Treatment: 20,693 | Control: 21,219

Device Type: DesktopDesktop
Average Lift
Revenue per Visit
(RPV)
Conversion Rate
(CR)
Average Order Value
(AOV)
+1.71%-1.19%+2.82%

Total Visitors: Treatment: 22,406 | Control: 20,436

Distribution of Average RPV Lift for All Visitors

Distribution of Average RPV Lift - Hide Chat in Cart and Checkout

The results distribution reveal that most merchants (83.7%) saw a positive RPV (2%). The results suggest the potential for greater lift for mobile users.

*These results reflect the average findings for all participating merchants and are not guaranteed for all users.

Data Analysis: Hide Chat in Cart and Checkout Experiment

What It Means

What It Means

As with other experiments that aim to reduce distractions on the cart and checkout pages, this experiment performs particularly well on mobile. This affirms other community findings that a simplified checkout funnel can have an impact on revenue per visitor (RPV), especially on mobile. Mobile treatments saw conversion rate and average order value lifts of +11.47% and +3.72% respectively.

Things to Consider

Chat is still an important function and should appear prominently in other areas of the eCommerce site. However, early results from this experiment show that it could be a distracting feature on the cart and checkout pages. As more merchants experiment with this treatment, we will examine the results based on data points such as real chat utilization, average time in checkout, and so on.

Who Should Try It

This experiment is recommended for all merchants. However, no two merchants are the same. If a merchant experiences high user engagement with the chat functionality, they should dig into their chat data. If the data indicates that chat is having an impact on conversions on the cart and checkout pages, approach this experiment with caution.

Who Should Try It

Desktop visitors respond particularly well to this treatment. This could be because the elements on a desktop site don’t have to compete as much for space or attention as those on a smaller mobile screen.

Interested in Learning More?

Get deeper insights into over 130 experiments available through the Initiative.