Pattern #2: Icon Labels

Pattern #2  Tested 8 timesFirst tested by James Foster Recently tested by Kyle Phillips on Jan 25, 2021

Based on 8 Tests, Members See How Likely Version B Wins Or Loses And By How Much

LOSSES
-5
-4
-3
-2
-1
FLAT
+1
+2
+3
+4
+5
WINS

Measured by the sum of negative and positive tests.

A B
Icon Labels (Variant A) Icon Labels (Variant B)

Expected Median Effects Of B

?

Progression

(8 tests)

-

Leads

-

Signups

-

Engagement

?

Sales

(2 tests)

-

Revenue

-

Retention

-

Referrals

?

ANY PRIMARY

(8 tests)

Tested on

Tests

Pattern #2: Icon Labels
Was Tested On Support.microsoft.com by Ronny Kohavi

Added

Isolated

Test # 216 on Support.microsof... by $conducted_test->test->user_->first_name . ' ' . $conducted_test->test->user_->last_name Ronny Kohavi    Dec 21, 2018 Test link

Find Out How It Did

  • Measured by survey completions

Zoom In
A
B
Home & Landing

Microsoft ran an experiment on their Customer Satisfaction Survey at both support.microsoft.com and answers.microsoft.com (Desktop). The treatment contained two icon labels at the opposite sides of the star rating range (ex: Very Dissatisfied and Very Satisfied) - providing it with additional meaning.

Get Access To See The Test Results

The Same Pattern Was Also Tested Here

Added

Isolated

Test # 334 on Thomasnet.com by $conducted_test->test->user_->first_name . ' ' . $conducted_test->test->user_->last_name Kyle Phillips    Jan 25, 2021 Test link

Find Out How It Did With 60,466 Visitors

  • Measured by clicks on any of the 3 nav links

Zoom In
  • ?% (? successes out of ? visits)
A
  • ?% (? successes out of ? visits)
B
Global

This experiment measured the impact of adding text labels to three icon-only nav items. 

Get Access To See The Test Results

Added

Isolated

Test # 211 Skype App by $conducted_test->test->user_->first_name . ' ' . $conducted_test->test->user_->last_name Ronny Kohavi    Nov 20, 2018 Test link

Find Out How It Did

  • Measured by calls being made

Zoom In
A
B
Global

Microsoft Skype ran an experiment for the mobile segment of the Skype application with a treatment having combined icons with corresponding labels. The control only showed icons.

Get Access To See The Test Results

Replaced

Isolated

Test # 190 on Diamondcandles.c... by $conducted_test->test->user_->first_name . ' ' . $conducted_test->test->user_->last_name Peep Laja    Jul 26, 2018 Test link

Find Out How It Did With 95,776 Visitors

  • Measured by clicks on menu

  • Measured by sales

Zoom In
  • ?% (? successes out of ? visits)
  • ?% (? successes out of ? visits)
A
  • ?% (? successes out of ? visits)
  • ?% (? successes out of ? visits)
B
Global

This test has explored numerous hamburger menu variations and has been covered in detail over at https://conversionxl.com/blog/testing-hamburger-icon-revenue/ - Thanks Peep Laja for sharing. Here we reported on a consistent increase in both menu clicks and sales.

Get Access To See The Test Results

Replaced

Isolated

Test # 191 on Diamondcandles.c... by $conducted_test->test->user_->first_name . ' ' . $conducted_test->test->user_->last_name Peep Laja    Jul 26, 2018 Test link

Find Out How It Did With 96,195 Visitors

  • Measured by clicks on menu

  • Measured by sales

Zoom In
  • ?% (? successes out of ? visits)
  • ?% (? successes out of ? visits)
A
  • ?% (? successes out of ? visits)
  • ?% (? successes out of ? visits)
B
Global

Get Access To See The Test Results

Replaced

Isolated

Test # 16 on Caffeineinformer... by $conducted_test->test->user_->first_name . ' ' . $conducted_test->test->user_->last_name James Foster    Jul 01, 2016 Test link

Find Out How It Did With 25,584 Visitors

  • Measured by clicks on button

Zoom In
  • ?% (? successes out of ? visits)
A
  • ?% (? successes out of ? visits)
B
Home & Landing

Get Access To See The Test Results

Replaced

Isolated

Test # 17 on Caffeineinformer... by $conducted_test->test->user_->first_name . ' ' . $conducted_test->test->user_->last_name James Foster    Jul 01, 2016 Test link

Find Out How It Did With 25,344 Visitors

  • Measured by clicks on button

Zoom In
  • ?% (? successes out of ? visits)
A
  • ?% (? successes out of ? visits)
B
Home & Landing

Get Access To See The Test Results

Replaced

Isolated

Test # 18 on Caffeineinformer... by $conducted_test->test->user_->first_name . ' ' . $conducted_test->test->user_->last_name James Foster    Jul 01, 2016 Test link

Find Out How It Did With 241,695 Visitors

  • Measured by clicks on button

Zoom In
  • ?% (? successes out of ? visits)
A
  • ?% (? successes out of ? visits)
B
Home & Landing

This is a retest of Test017 with a lengthier testing duration.

Get Access To See The Test Results

Leaks

Leak #14 from Google.com   |   Jul 1, 2019 Listing

Google's A/B Test Is Hinting That Labels With Icons Are Better Than Labels Alone

A - May 15, 2019
B - May 15, 2019 IMPLEMENTED

Take a look at these two screenshots, both taken on May 15, 2019. With some cookie clearing magic in between it's pretty clear that Google ran an icon experiment on their search results page. The control version had labels only in the top nav, whereas the variant contained icons in addition to the labels. The variation was implemented a month later in June. View Leak

+0.5 Evidence

Leak #17 from Booking.com   |   Jul 30, 2019 Home & Landing

Booking Also A/B Tests Icon Labels Replicating Google's Recent Experiment

A - Jul 18, 2019
B - Jul 18, 2019 IMPLEMENTED

It seems like Booking just completed running an experiment that is very similar to a recent one from Google. The experiment evaluated icons with labels in the top navigation against a control version with labels only.  View Leak

+0.5 Evidence

For each pattern, we measure three key data points derived from related tests:

REPEATABILITY - this is a measure of how often a given pattern has generated a positive or negative effect. The higher this number, the more likely the pattern will continue to repeat.

SHALLOW MEDIAN - this is a median effect measured with low intent actions such as initiating the first step of a lengthier process

DEEP MEDIAN - this is derived from the highest intent metrics that we have for a given test such as fully completed signups or sales.