Pattern #1: Remove Coupon Fields

Pattern #1  Tested 6 timesFirst tested by Herman Klein Recently tested by Melina Hess on Jan 19, 2022

With A Total Of 2 Evidence For Version B It Is Likely Better

LOSSES
-5
-4
-3
-2
-1
FLAT
+1
+2
+3
+4
+5
WINS

Measured by the sum of negative and positive tests.

A B
Remove Coupon Fields (Variant A) Remove Coupon Fields (Variant B)

Expected Median Effects Of B

6%

Progression

(1 tests)

-

Leads

-

Signups

-

Engagement

2.4%

Sales

(5 tests)

15%

Revenue

(2 tests)

-

Retention

-

Referrals

2.5%

ANY PRIMARY

(6 tests)

Tested on

Tests

Pattern #1: Remove Coupon Fields
Was Tested by Someone

Replaced

Isolated

Test # 274 by $conducted_test->test->user_->first_name . ' ' . $conducted_test->test->user_->last_name Someone    Dec 16, 2019 Test link

Variation B Was Insignificant

+0 Evidence

  • Measured by completed transactions   |   p-val 0.6155043

Zoom In
  • 80.9% (1590 successes out of 1965 visits)
A
  • 81.5% (1538 successes out of 1886 visits)
B
Checkout

In this experiment, a fully visible coupon field (A) was made less visible by turning it into a default collaped link (B). Clicking on the link caused the coupon field to appear.

Replaced

Isolated

Test # 393 on Snocks.com by $conducted_test->test->user_->first_name . ' ' . $conducted_test->test->user_->last_name Melina Hess    Jan 19, 2022 Test link

Variation B Was Likely Worse

-0.5 Evidence

  • Measured by completed transactions   |   p-val 0.06906723

Zoom In
  • 59.4% (9922 successes out of 16712 visits)
A
  • 58.4% (9847 successes out of 16863 visits)
B
Shopping Cart

This experiment replaced a visible coupon field with a dynamic text link that would initially hide the form field. Only after clicking the text link would the coupon form field appear. The translation from German is "Do you have a coupon code? Click here to apply". Impact on completed transactions was measured.

Replaced

Isolated

Test # 389 on Svsound.com by $conducted_test->test->user_->first_name . ' ' . $conducted_test->test->user_->last_name Keenan Davis    Dec 16, 2021 Test link

Variation B Was Insignificant

+0 Evidence

  • Measured by total revenue   |   p-val 0.087

  • Measured by completed sales   |   p-val 0.28780505

Zoom In
  • 17.3% (1761 successes out of 10188 visits)
A
  • 17.9% (1809 successes out of 10133 visits)
B
Checkout

In this simple experiment, a highly visible coupon field was replaced with a less visible (but clickable) link in the variation. Clicking on the link would show the coupon field. Impact on sales and revenue was measured.

Replaced

Isolated

Test # 145 on Normanrecords.co... by $conducted_test->test->user_->first_name . ' ' . $conducted_test->test->user_->last_name Nathon Raine    Jan 18, 2018 Test link

Variation B Was Likely Better

+0.5 Evidence

  • Measured by completed sales   |   p-val 0.20762012

Zoom In
  • 72.0% (1502 successes out of 2086 visits)
A
  • 73.7% (1514 successes out of 2053 visits)
B
Checkout

In this test the coupon field was replaced with a small link that would bring the field back if needed. This is a more suble approach than just completely removing the coupon field. It still allows for the use of coupon fields by those customers which are truly searching for a way to enter their aquired codes.

Removed

Isolated

Test # 42 on Adoramapix.com by $conducted_test->test->user_->first_name . ' ' . $conducted_test->test->user_->last_name Herman Klein    May 11, 2016 Test link

Variation B Was Better

+1 Evidence

  • Measured by visits to shopping cart   |   p-val 0.00000001

  • Measured by post-purchase page visits   |   p-val 0.01276806

Zoom In
  • 72.8% (5576 successes out of 7655 visits)
  • 70.1% (5365 successes out of 7655 visits)
A
  • 77.2% (6072 successes out of 7861 visits)
  • 71.9% (5652 successes out of 7861 visits)
B
Shopping Cart

Removed

Isolated

Test # 121 on Bionicgloves.com by $conducted_test->test->user_->first_name . ' ' . $conducted_test->test->user_->last_name VWO    Mar 13, 2015 Test link

Variation B Was Better

+1 Evidence

Zoom In
A
B

Source: vwo.com/blog/promo-code-box-ecommerce-website-bleeding-dollars-ab-test/

Shopping Cart

VWO.com published this test which removed two coupon fields on a shopping cart: a gift card code and a special offer code.

Related Patterns

For each pattern, we measure three key data points derived from related tests:

REPEATABILITY - this is a measure of how often a given pattern has generated a positive or negative effect. The higher this number, the more likely the pattern will continue to repeat.

SHALLOW MEDIAN - this is a median effect measured with low intent actions such as initiating the first step of a lengthier process

DEEP MEDIAN - this is derived from the highest intent metrics that we have for a given test such as fully completed signups or sales.