Impact of usability monitoring
influencing product and business decisions
Build a program for continuous monitoring
Developed a systematic approach to evaluate and address product experience health (usability) issues, integrating insights from various sources including in-app VOC data, heuristics, analytics, and usability research. Identified usability issues are logged into the product team’s backlog for prioritization. Utilizing a visual journey map, all issues are mapped for each product or feature set, showcasing data sources and facilitating identification of interconnected issues requiring cross-functional collaboration. Prioritized critical issues are grouped and addressed by specialized "tiger teams" or initiative groups, promoting comprehensive resolution.
Over 70% of McAfee employees engage with the UX - product health channel, with 30% actively contributing insights. The approach is exemplified through three case studies demonstrating how the research team effectively supported product and business objectives through usability investigations.
Approach to improving usability
There are 3 case studies below of how the research team helped achieve product and business targets through usability investigations.
Case Study 1:
Users are paying for different stand-alone products but not using all of them. Why?
Method
Used a heuristic evaluation (PURE method) to identify issues, prioritize them and drive discussion with Designers, PM, and Dev on how to include them in the roadmap.
Data
Customer support call volumes around authentication jumped. Lots of virulent VOC comments surfaced. UXR jumped in and poked around.
The heuristic evaluation and analytics data revealed a pattern of pain points around authentication.
Users had to authenticate each time as they activated new features within the McAfee security suite.
Authentication required entering an OTP sent to their email or phone. Analytics data showed that 19% of the OTP requests did not get validated.
Insight: The user’s mental model differs from the product interaction flow.
User’s mental model
The user is buying from the brand, McAfee. So even when the user buys what the business thinks are different products, to the user s/he is buying different features from one brand. the user thinks of it as buying a suite of features within one product. Naturally, s/he expects to log in and authenticate only once to access all the features. Especially, if all the suite of features is on the same device.
How the product behaves
As the user moves between different products which are loosely integrated, s/he is asked to authenticate each time. This causes great user friction and is a barrier to engagement and adoption.
Action
An initiative was added to the roadmap to allow for single-sign-on and one-time authentication across all different reatures within the McAfee security suite.
This initiative will be expanded to allow single-sign-on across devices for users who have McAfee products on multiple PCs and phones.
Case Study 2:
Why are more users not setting up the cool, new Dark Web Scan capability?
Umm…UXR ..do you know?
Method
Creating a user journey map using analytics, digging into the why through in-app surveys and usability studies.
Data
The steps to use it is long and laborious: 70% of users are dropping off after step 20!
Users really don’t understand the value of Dark Web Scan. Users entering the product from certain channels, are not seeing the onboarding flow for the specific feature. For the users skipping the onboarding flow, we haven’t done a great job of answering - How does it protect me?
Action
After 3 sprint cycles, many of the kinks are ironed out. In the next 8 weeks, we watched the count for users who set up jumped 5X from 10,000 to 50,000.
Case Study 3:
Why aren’t our users engaging with the breach alert information we are presenting? Don’t they care?
Method
The team created a customer journey map using a heuristic evaluation and usability testing results.
Insights
When testing this new feature, we discovered that many users preferred entering old, less often used email IDs. The users had numerous accounts associated with these old email accounts. However, when they ran the breach scan and encounterd a long list of breaches, users experienced cognitive overload, feeling paralyzed and unsure of how to proceed. Recognizing a gap in our solution, we found that users needed to leave the app to address breach issues, leading to high friction and disengagement. Consequently, many users closed the breach list without taking action, with most rarely revisiting the process.
Action
Applying our knowledge of what types of information users want to protect the most, we now show a shorter list of breaches. Users feel they can make a dent on their list of breaches so they take action.
Automatic breach remediation is a difficult technical challenge. The team has taken some baby steps towards it. We take the user’s permission to delete accounts that have been breached. This way, the user does not have to take the action themselves. More importantly, we don’t ask them to figure out how to take action outside our product, in an unpredictable environment.
A zoomed in view of the journey map above
