Pattern #52: How It Works

Pattern #52  Tested 4 timesFirst tested by Justin Baer Recently tested by Michael McSweeney on Nov 28, 2023

Based on 4 Tests, Members See How Likely Version B Wins Or Loses And By How Much

LOSSES
-5
-4
-3
-2
-1
FLAT
+1
+2
+3
+4
+5
WINS

Measured by the sum of negative and positive tests.

A B
How It Works (Variant A) How It Works (Variant B)

Expected Median Effects Of B

?

Progression

(1 tests)

?

Leads

(1 tests)

?

Signups

(1 tests)

-

Engagement

?

Sales

(3 tests)

-

Revenue

-

Retention

-

Referrals

?

ANY PRIMARY

(4 tests)

Tested on

Tests

Pattern #52: How It Works
Was Tested On Shmoodyapp.com by Michael McSweeney

Replaced

Isolated

Test # 505 on Shmoodyapp.com by $conducted_test->test->user_->first_name . ' ' . $conducted_test->test->user_->last_name Michael McSweeney    Nov 28, 2023 Test link

Find Out How It Did With 8,231 Visitors

  • Measured by transactions (7 days later)

Zoom In
  • ?% (? successes out of ? visits)
A
  • ?% (? successes out of ? visits)
B
Signup

In this experiment, new paywall copy and messaging was used to encourage more users to signup and pay for access to a mental health app. The variation reinforced that users would be receiving a full featured application, with reminders about the upcoming billing. It reinforced that users will have a full week to decide and potentially cancel their application. Impact on lagging +7 day later transactions was measured.

Get Access To See The Test Results

The Same Pattern Was Also Tested Here

Added

Isolated

Test # 395 by $conducted_test->test->user_->first_name . ' ' . $conducted_test->test->user_->last_name Jakub Linowski    Jan 31, 2022 Test link

Find Out How It Did With 17,099 Visitors

  • Measured by adds to cart

  • Measured by completed sales

Zoom In
  • ?% (? successes out of ? visits)
  • ?% (? successes out of ? visits)
A
  • ?% (? successes out of ? visits)
  • ?% (? successes out of ? visits)
B
Product

In this experiment, a How It Works section was appended on a product detail page just above product reviews.

Get Access To See The Test Results

Removed

Isolated

Test # 200 on Trydesignlab.com by $conducted_test->test->user_->first_name . ' ' . $conducted_test->test->user_->last_name Will Anderson    Sep 14, 2018 Test link

Find Out How It Did With 33,802 Visitors

  • Measured by completed signups

  • Measured by course enrolments (paid)

Zoom In
  • ?% (? successes out of ? visits)
  • ?% (? successes out of ? visits)
A
  • ?% (? successes out of ? visits)
  • ?% (? successes out of ? visits)
B
Home & Landing

In this experiment, a "How It Works" content section was removed.

Get Access To See The Test Results

Added

Isolated

Test # 23 on Checkineasy.com by $conducted_test->test->user_->first_name . ' ' . $conducted_test->test->user_->last_name Justin Baer    Oct 01, 2015 Test link

Find Out How It Did With 4,743 Visitors

  • Measured by visits to post-signup

Zoom In
  • ?% (? successes out of ? visits)
A
  • ?% (? successes out of ? visits)
B
Home & Landing

Get Access To See The Test Results

For each pattern, we measure three key data points derived from related tests:

REPEATABILITY - this is a measure of how often a given pattern has generated a positive or negative effect. The higher this number, the more likely the pattern will continue to repeat.

SHALLOW MEDIAN - this is a median effect measured with low intent actions such as initiating the first step of a lengthier process

DEEP MEDIAN - this is derived from the highest intent metrics that we have for a given test such as fully completed signups or sales.