Pattern #8: Natural Language Forms

Pattern #8  Tested 5 timesFirst tested by Greg Van Horn Recently tested by Patrick McKenzie on Aug 26, 2016

Based on 5 Tests, Members See How Likely Version B Wins Or Loses And By How Much

LOSSES
-5
-4
-3
-2
-1
FLAT
+1
+2
+3
+4
+5
WINS

Measured by the sum of negative and positive tests.

A B
Natural Language Forms (Variant A) Natural Language Forms (Variant B)

Expected Median Effects Of B

?

Progression

(2 tests)

?

Leads

(4 tests)

?

Signups

(1 tests)

-

Engagement

-

Sales

-

Revenue

-

Retention

-

Referrals

?

ANY PRIMARY

(5 tests)

Tested on

Tests

Pattern #8: Natural Language Forms
Was Tested On Embracepetinsurance.com by Matt Aster

Replaced

Isolated

Test # 15 on Embracepetinsura... by $conducted_test->test->user_->first_name . ' ' . $conducted_test->test->user_->last_name Matt Aster    Aug 28, 2015 Test link

Find Out How It Did With 18,638 Visitors

  • Measured by policy selection start

  • Measured by visits to quote screen

Zoom In
  • ?% (? successes out of ? visits)
  • ?% (? successes out of ? visits)
A
  • ?% (? successes out of ? visits)
  • ?% (? successes out of ? visits)
B
Home & Landing

Get Access To See The Test Results

The Same Pattern Was Also Tested Here

Replaced

Isolated

Test # 14 on Prizegrab.com by $conducted_test->test->user_->first_name . ' ' . $conducted_test->test->user_->last_name Greg Van Horn    Oct 20, 2016 Test link

Find Out How It Did With 3,133 Visitors

  • Measured by form submits

Zoom In
  • ?% (? successes out of ? visits)
A
  • ?% (? successes out of ? visits)
B
Home & Landing

Get Access To See The Test Results

Replaced

Isolated

Test # 58 on Bingocardcreator... by $conducted_test->test->user_->first_name . ' ' . $conducted_test->test->user_->last_name Patrick McKenzie    Aug 26, 2016 Test link

Find Out How It Did With 736 Visitors

  • Measured by visits to post-signup

Zoom In
  • ?% (? successes out of ? visits)
A
  • ?% (? successes out of ? visits)
B
Home & Landing

Get Access To See The Test Results

Replaced

Isolated

Test # 60 on Embracepetinsura... by $conducted_test->test->user_->first_name . ' ' . $conducted_test->test->user_->last_name Matt Aster    Sep 30, 2015 Test link

Find Out How It Did With 12,102 Visitors

  • Measured by policy selection start

  • Measured by visits to quote screen

Zoom In
  • ?% (? successes out of ? visits)
  • ?% (? successes out of ? visits)
A
  • ?% (? successes out of ? visits)
  • ?% (? successes out of ? visits)
B
Home & Landing

Get Access To See The Test Results

Leaks

Leak #68 from Airbnb.com   |   Apr 20, 2021 Home & Landing

Airbnb A/B Tests And Rejects A Natural Language Form

A - Mar 18, 2021
B - Mar 18, 2021 REJECTED

Airbnb just completed an interesting experiment on their Become A Host landing page. They replaced a simple headline with a more dynamic natural language form that provided a sense of how much money someone could make from hosting their place. View Leak

-0.5 Evidence

For each pattern, we measure three key data points derived from related tests:

REPEATABILITY - this is a measure of how often a given pattern has generated a positive or negative effect. The higher this number, the more likely the pattern will continue to repeat.

SHALLOW MEDIAN - this is a median effect measured with low intent actions such as initiating the first step of a lengthier process

DEEP MEDIAN - this is derived from the highest intent metrics that we have for a given test such as fully completed signups or sales.