A/B User Testing for Improved Navigation

macbook-pro-clay copy.png

Context

This was a usability test for a life insurance tech start up’s (specific name available upon request) Core Web App's top navigation bar. I examined the discoverability of the dashboard/home page with or without an explicit "dashboard" label. And to research if most users would know to use the logo in the absence of a label as a return to the home page/dashboard.

Key Assumptions

Test A: We expect that the label will make it easier for users to find the dashboard.

Test B: We expect that the user will recognize the logo as an area to return to the dashboard.

 

Key Questions

We will use an A/B test to see which top navigation best helps users find the dashboard. Question for user was : Where will you click to return to the home page?

 

Key Outcomes/Metrics

  • Time to complete

  • Statistically significant difference between designs

The tool used was Usabilityhub and below are heat maps of users click paths. I used MeasuringU's Binomial Test with Confidence Interval to determine statistically significant differences between the design patterns.

test a.png

Test A

When both the "dashboard" label and logo were included, we can be 80% confident in the assumption that the logo still worked as a return to home page even despite the presence of the label "Dashboard." (Average response time 48.6 seconds).

Group 18.png

Test B

When we removed the "Dashboard" label, 100% of the participants selected the logo as a return to the dashboard. (Average response time 40.7 seconds).

 

Results

When all participants are calculated together we can still be 90% confident in the logo as the primary click path back to the home page. The majority of written responses from participants also supported the users' understanding that the logo worked as a return to the home page/dashboard.

"Usually, when there isn't a distinct home button, I click on the site name, which is often in the upper left hand corner to take me back to the home page."

"I was recently discussing this with a friend. - We both agreed it was VERY NORMAL and almost expected, that you can click the logo to return to the home page."

"There is no explicit home page link, so clicking on the logo of the page is a normal way to return home."

"Most websites use their logo as a way to return to the homepage."

 

Design Recommendations

Use the logo as a return to the home page/dashboard for users to click the correct part of the top navigation. This will also declutter the top navigation bar.