Pattern #6: Customer Star Ratings

Pattern #6  Tested 6 timesFirst tested by Someone Recently tested by Katharina Lay on Aug 12, 2024

Based on 6 Tests, Members See How Likely Version B Wins Or Loses And By How Much

LOSSES
-5
-4
-3
-2
-1
FLAT
+1
+2
+3
+4
+5
WINS

Measured by the sum of negative and positive tests.

A B
Customer Star Ratings (Variant A) Customer Star Ratings (Variant B)

Expected Median Effects Of B

?

Progression

(2 tests)

-

Leads

-

Signups

?

Engagement

(1 tests)

?

Sales

(4 tests)

-

Revenue

-

Retention

-

Referrals

?

ANY PRIMARY

(6 tests)

Tested on

Tests

Pattern #6: Customer Star Ratings
Was Tested On Aboalarm.de by Katharina Lay

Added

Isolated

Test # 547 on Aboalarm.de by $conducted_test->test->user_->first_name . ' ' . $conducted_test->test->user_->last_name Katharina Lay    Aug 12, 2024 Test link

Find Out How It Did With 102,221 Visitors

  • Measured by paid signups

Zoom In
  • ?% (? successes out of ? visits)
A
  • ?% (? successes out of ? visits)
B
Signup

In this 4 step signup funnel experiment, social proof and customer reviews were appended at the bottom of the screen. Impact on conversions was measured.

Get Access To See The Test Results

The Same Pattern Was Also Tested Here

Replaced

Confounded

Test # 468 on Umbraco.com by $conducted_test->test->user_->first_name . ' ' . $conducted_test->test->user_->last_name Lars Skjold Iversen    Apr 28, 2023 Test link

Find Out How It Did With 45,082 Visitors

  • Measured by clicks to book a demo

Zoom In
  • ?% (? successes out of ? visits)
A
  • ?% (? successes out of ? visits)
B
Home & Landing

In this homepage experiment a number of changes were introduced - with perhaps the most prominent one being the replacement of a video component with customer review badges. Additional copy changes included reinforcement of the latest version number (v11) throughout the page, as well as a dedicated (v11) section in the middle of the page. Impact on demo signups was measured.

Get Access To See The Test Results

Replaced

Isolated

Test # 232 on Yoast.com by $conducted_test->test->user_->first_name . ' ' . $conducted_test->test->user_->last_name Sjardo Janssen    Mar 15, 2019 Test link

Find Out How It Did With 20,661 Visitors

  • Measured by total sales

Zoom In
  • ?% (? successes out of ? visits)
A
  • ?% (? successes out of ? visits)
B
Checkout

In this experiment, the Yoast team replaced one of the bulleted reassurances on their checkout page ("4.9 out of 5 stars" vs "24/7 Free support"). Raising the question - is free support or high reviews valued more? - Thanks Sjardo & Meike for sharing!

Get Access To See The Test Results

Added

Isolated

Test # 182 on Yummly.com by $conducted_test->test->user_->first_name . ' ' . $conducted_test->test->user_->last_name Marcos Ciarrocchi    Jun 12, 2018 Test link

Find Out How It Did With 673,296 Visitors

  • Measured by recipe page opens

Zoom In
  • ?% (? successes out of ? visits)
A
  • ?% (? successes out of ? visits)
B
Listing

Get Access To See The Test Results

Added

Isolated

Test # 43 on Embracepetinsura... by $conducted_test->test->user_->first_name . ' ' . $conducted_test->test->user_->last_name Matt Aster    Aug 01, 2015 Test link

Find Out How It Did With 69,436 Visitors

  • Measured by visits to thank you page

Zoom In
  • ?% (? successes out of ? visits)
A
  • ?% (? successes out of ? visits)
B
Home & Landing

Get Access To See The Test Results

Leaks

Leak #48 from Airbnb.com   |   Mar 9, 2020 Listing

Airbnb A/B Tests And Implements Customer Rating Averages Above Search Results

A - Feb 5, 2020
B - Feb 5, 2020 IMPLEMENTED

In this simple experiment, Airbnb tested the display of customer rarings averages on their search results screen. The tested copy was a form of social proof which showed the average rating of the properties for a given city (defined by the search). View Leak

+0.5 Evidence

Leak #26 from Airbnb.com   |   Oct 3, 2019 Product

Airbnb Discovers A Better Way To Display Customer Ratings In This A/B Test

A - Sep 3, 2019
B - Sep 3, 2019 IMPLEMENTED

In this tiny and well isolated experiment, Airbnb tried a slightly different approach to displaying customer ratings on their property (product) pages. From the observed implementation decision one month later, we learned that using decimal places for customer rating averages seems to be the better approach. :)  View Leak

+0.5 Evidence

For each pattern, we measure three key data points derived from related tests:

REPEATABILITY - this is a measure of how often a given pattern has generated a positive or negative effect. The higher this number, the more likely the pattern will continue to repeat.

SHALLOW MEDIAN - this is a median effect measured with low intent actions such as initiating the first step of a lengthier process

DEEP MEDIAN - this is derived from the highest intent metrics that we have for a given test such as fully completed signups or sales.