Netflix A/B Tests And Rejects Secondary Sign-in And Sign-up Calls To Action

In the B variation we can see that a text with a link "Have an account? Sign In" has been added as an alternative call to action. And yet this variation did not outperform the control version without the added link. I find this little piece of evidence interesting as it's a subtle challenge to the Hobson’s+1 Choice Effect - Bart Schutz's general idea of having more than one call to action. It looks like having a single call to action in this case worked out better. Could it be that the alternative sign in link distracted people from signing up and lead to an increase in additional errors?

A possibly better way to do sign-in

If the above explanation holds, then it might be inline with a pattern about combining sign-in and signup (from a well respected experimenter - Ethan Smith). The idea is to merge signing in and signing up by combining enough smarts to detect whether it's a return signin or first time signup. Although we still don't have any real data for or against this pattern.

So does this mean that all secondary choices won't work?

Not at all. We actually do have some accumulating evidence in favor of showing alternative links that educate users about a product or service (think "Learn More" links and such). The idea is to offer secondary links to pages with more information (and repated calls to action) that help them choose whether the product or service is for them or not.

In the C variant we can see a secondary message with the text and link "Been a member before? Restart membership". This also didn't outperform the control variation. Could this simply be because this secondary link was actually artificial? There was no logic or personalization which goverened it being displayed. It was simply a shot in the dark with the hope of perhaps connecting with users who might or might not have been members in the past. Well, this didn't work.

And yet personalized messaging for return visitors does show promse

When we look at example of screens that have been adjusted for return users based on "real unfinished tasks", we see positive evidence in favor of such patterns as cart reminders and personalized buttons. Return visit personalization can work. it just needs to be based on real user actions and not artificial guesses.