I’ve written about a couple of example cases and explained how onboarding bandaids often don’t work. Time to dive into the top of the marketing funnel.

Identify the Right Problem

I love challenging assumptions, so when I am told there is a problem and the cause is “that part”, I always take a step back and consider the system. As I wrote in part 2, many folks thought we had a problem with the social platform’s homepage since so many new users did not return. Since the homepage was something each of their experiences had in common, you can see how the conclusion makes sense.

However, we had little insight beyond web analytics. I did what I always do to understand what’s going on with underperforming products – I conducted user research in the form of a usability test. One testing approach would have focused on the homepage and I could have designed a test that only showed participants the homepage and evaluated how well it communicated the site’s purpose, or how easy it was to navigate, or determined new users’ initial actions, or other things. However, no one shows up at the homepage without signing up and no one signs up without first loading our landing page and they don’t get there without responding to an email or mailing from one of our partners.

All of that context is crucial to setting up the study to answer the question: Why are new users not engaging?

So, how do you communicate that context? You recruit representative users and have them experience all those steps themselves. When they reach the homepage, they will have the same knowledge and context as the people you hope to better understand.

The Problem

As I wrote earlier, it was quickly obvious that the initial messaging was confusing and misleading. The biggest problem was terminology which is not too surprising when you consider an audience from across the country with varying backgrounds.

Here’s just one example of many: The platform offered discussion forums – topic-based areas where members could post messages and comment on those messages. However, the initial email and landing page simply called these “groups”. Some participants thought these were membership-based clubs that met on a regular schedule and required a level of commitment and some even extrapolated from there to assume groups only met in-person.

This and other miscommunications affected participants differently. Some became interested in things the platform didn’t offer. Some became less engaged due to a misunderstanding. Others would have been more engaged had they properly understood. We even found that a strong emphasis on “making friends” had a surprisingly negative effect on many people we interviewed. So, viewing the homepage with these varied understandings was quite different than landing there without preconceptions. Assumptions and first impressions had been formed and they affected behavior.

Even though the stated goal for improving the homepage and onboarding was increasing engagement, I don’t think we could meaningfully do that without tackling this problem first. We really needed to reduce the number of variables we were designing for. Without upstream changes, we’d have to tackle these misunderstandings first in the product as part of onboarding.

This is why aligning marketing, business development, and product was the first priority. Even our small company had built silos that needed to be breached.

Fixing the Funnel

Our testing and research helped us better understand members’ priorities and the language they naturally used for the concepts we offered. Identifying the problem also gave us a wealth of knowledge we could use in designing the solution and, later, improving the homepage and the platform as a whole.

We shared our findings with the entire company, sharing test video to really prove the point. That got everyone on board to work on solving the problem.

With a clearly defined problem, design follows easily. 

Our design team worked collaboratively with other teams to iteratively redesign the marketing messaging and landing page. We improved and aligned terminology, we added adjectives to clearly communicate concepts, and we made sure to reflect the user priorities we heard in our research. We kept testing, too, and could see the improvement after each iteration. 

After the next major release, we found these design changes doubled the conversion rate.

It was a successful project and not at all what stakeholders thought they were asking for.

---

Thanks for reading. Stay tuned for the final part of the series where I discuss addressing core design problems affecting a released product.

This is part 3 of my blog series on onboarding and retention.

When new users struggle with a product, there are a few quick fixes teams often try. They are simple, cheap (sometimes), and have low product impact (there is little code needed). I’ll take a look at a few and point out some gotchas.

More Explanation

A common fix is to throw more information at the user.

The explanations come in many forms:

• Additional documentation, maybe with links embedded in various parts of the application
• Blocks of explanatory text embedded on the app pages (in-product documentation)
• Marketing email campaigns
• Outreach from customer support to help users one-on-one

These are pretty blunt instruments. They rely on your users reading or watching additional content. It’s a common refrain that “people don’t read” and that is often true. It’s especially true if the person is not motivated to do it. In trial or try out cases, the user is often looking to see how your product is going to help them, not read more about it.

In addition, these approaches have other drawbacks. In-product documentation engagement is hard to measure. Marketing emails are generally better at getting people back into the product than explaining concepts. Customer support is likely effective if they reach a user but is a very expensive and non-scalable solution.

Finally, in-product and standalone documentation are pretty static. If you’re not careful, you will end up with blocks of text that, once read, provide zero to negative value. Or, you will introduce a ton of documentation that few people read and must be maintained which, for a quickly evolving product, is not an easy task.

Tooltips, Product Tours, “In-App Experiences”

I am sure you’ve experienced the tooltip onboarding aid. Your first visit to an app pops up a number of small, helpful tips that point things out on-screen or guide you through one or more tasks. There’s no doubt these can be helpful and multiple companies have built tool businesses around this pattern.

The many 3rd party products are easy to integrate into applications and, better yet, enable anyone outside the engineering team to author and maintain the tooltips. However, authors need to take care that their tooltips and flows are well-tested and understand that every app update risks breaking them. The tooltip triggers and positioning often rely on HTML and CSS details that, if poorly understood, can result in surprising behavior. Poorly engineered front-end markup can also make for brittle tooltips.

Another weakness I’ve seen with these tools (though, perhaps someone has addressed this) is that it is very difficult to support guided flows across multiple page and state changes. For example, the tooltip logic might rely on a list containing only a single item and will only work the first time a user does the associated task. Or you may want tooltips to appear only if the application is in a certain state but that data is not available to the tooltip logic. These constraints force the author to only provide guidance in specific, limited contexts.

While each tool has its own technical strengths and weaknesses, the pitfalls to avoid are universal. Here are a few:

• Obvious explanations – Try to avoid tooltips that explain obvious concepts or operations. Is it really necessary to point out the logout button with a note saying it logs you out? Maybe in some cases, but error on the side of brevity.
• Excessive explanations – The social platform I described in part 2 had over 20 tooltips, displayed one at a time, that explained every button and section of the app’s home page. Torture. When we conducted usability testing, we found the homepage and main navigation was understandable and those tooltips were unnecessary.
• Bad timing – Timing is everything. Someone learning a new application will simply not absorb information that is provided at the wrong time or in the wrong context. For example, if you need to explain how to schedule a new blog post, don’t explain it when I create a new, empty post (or worse, right after I’ve created my account!). A better time would be when I save a draft or attempt to publish a post.
• Forced flows – Some guided flows take control away from the user, likely to avoid having the flow disrupted by an errant click or a bored user. Some force users down irrelevant (to them) paths. The social platform I worked on had at least a couple of distinct personas. If a new member, uninterested in social interactions, was forced down a guided tour about online chats and discussions, they may abandon the product. If the user is not engaged, they are unlikely to learn or remember these.
• Poor explanations – Sometimes a product using unfamiliar jargon or concepts is a design deficiency you need to bandaid. If the author is unaware, the help can continue obfuscating things. Instead, treat the tooltips as an opportunity to help new users translate familiar concepts and terms into the application’s world. Be sure to write tips in the user’s language, even if that departs from marketing, branding, and product jargon.

This type of help is not a magic bullet. To be successful, you need to do the work to understand where and when your users need help and apply it sparingly.

As a matter of fact, the work needed to design in-product help is a lot like the work needed to design a product… Hmm… 🤔

Helping People

For new users to your product, your goal is to make them proficient users of the product. For many products, these people will need to learn something. Jared Spool has a nice analogy describing this learning destination as “target knowledge”. Your goal is to get them from their current knowledge to this target knowledge. Ideally, you can do this as an implicit part of the product design by doing your research and designing it in.

• Know your users. Understand their starting knowledge, behaviors, and goals.
• Design for new and experienced users. I will touch on this more in part 5 where refactoring the design to better support new users actually helped existing users as much or more.
• Introduce concepts incrementally if you can. Provide help (in whatever form) when the user needs it.

You may still find areas where guides and other bandaids are needed but I bet you will be able to deploy them with precision and confidence.

Wrap Up

Like any product, feature, document, or anything with an audience, you need to do your research to understand how your users think, what their goals are, and what expectations and knowledge they come to you with. Without that work, even though these fixes don’t heavily impact the existing product, these aids will fail to help.

---

In parts 4 and 5, we’ll look at the cases I described in part 2. Both these examples illustrate cases where bandaids like these made little impact.

In part 1, I described how retention issues affect businesses and how UX teams are sometimes asked to fix the problems.

It’s story time! Let’s take a look at a couple of real-life examples.

A Social Platform

The first product facing challenges is a social platform aimed at older adults. Think Facebook in terms of general functionality with additional virtual, live programming. Membership was paid for by a sponsoring organization, such as a health insurance company, and provided to their customers as a free benefit.

Prospective new members were contacted by the sponsor via email or snail mail brochures. This material contained a link to a landing page with more info where the older adult could go on to register as a new member.

The product suffered from low retention rates and there were a bunch of theories as to why, ranging from “it looks too complicated” to “there’s not enough activity” to “members can’t remember how to get back to the site.” In particular, there was a lot of focus on the site’s homepage, suspected of being overwhelming, confusing, or otherwise unusable. Since this was a member’s first and subsequent view of the platform, it made sense to ensure it was as welcoming and clear as possible.

But, was the homepage the problem? It is very challenging to get any feedback from disengaged members, let alone something as nebulous as first impressions from a website visit.

To better understand what a new user experienced, I designed a user research study that started at the beginning – with a simulated email from the participant’s health insurance company. During the session, we observed participants read the email, visit the landing page, registration page, and the homepage.

Even before we had completed the 5 planned sessions, it was obvious that the homepage was not the main issue. We discovered that the sponsor’s email and our landing page both painted an ambiguous picture. Participants simply did not understand what was offered. Many asked, “Is this a dating site?” (It is not.) “Is this a group that meets in my area?” (It is purely virtual.) Or, they interpreted ambiguous terms in their own ways. For example, we started gaining insight on the various interpretations of “meeting people” or even “learning” – terms we thought were straight forward.

This research was crucial for two reasons:

1. It gave us valuable insight into how our audience interpreted our messaging, what terminology they used to describe things, and what resonated for them.
2. It challenged existing assumptions and pointed to a problem no one knew we had.

Before we could tackle engagement, we needed to ensure new members were set up for success. Focusing on acquisition first would also help increase the odds that people who may be a good fit would sign up in the first place.

Software QA Test Tool

The second product was a test automation tool for software teams. It had a 2 week free trial and, like any trial, the hope was trial users could quickly get up to speed, discover the product’s value, and become paying customers.

When I joined this project, the product was mostly built and nearing public release. My initial usability testing with new users discovered a number of usability issues. However, due to the impending release, we could make only superficial changes.

After release, conversation rates and feedback from Sales and Support confirmed we had a problem. Product leadership tasked the team to “fix the onboarding.”

We tried different approaches that avoided big product changes:

1. We put in a few subtle hint popups attempting to steer new users to the most relevant pages
2. We broke up the very first interaction into a few steps to provide more explanation and force a couple of actions we thought were necessary to fully evaluate the product
3. We added additional tooltip flows using Appcues to guide the new user through their first key tasks

While we saw limited success with each of these changes, they didn’t move the needle. Again, I continued usability testing to understand what trial users were doing and thinking. The results built on the initial testing. My initial conclusions clarified into an inescapable truth – the product had fundamental issues that no number of bandaids could fix.

New users were confronted with too many product features and concepts all at once. And, because of the design, there was no way to avoid or defer many of the most confusing or complicated aspects. As a result, many test participants failed to complete the most basic task a QA test tool should support – creating a test – during a test session. That is not a good first impression.

There was no way to “fix the onboarding” and we’d have to take a step back and figure out how to fix the product.

---

In part 3, I focus on the quick fixes we tried and their pitfalls. In parts 4 & 5, I’ll tell you more about how we tackled the problems described here.

“We have a user retention problem. They just aren’t coming back and it seems like new users don’t seem to know what to do. We need to improve our onboarding.”

I’ve heard this or something like it many times in my career.

The thinking goes like this: If users aren’t coming back then they must not be getting value. And if they aren’t getting value, that must mean that they couldn’t find the value. (We provide a lot of value!) If they can’t find it, then the onboarding is broken – it needs to better show them the value!

Quick Fixes

Chances are, if you’re experiencing retention problems, it’s a new product and your company is a startup. Startup culture is not afraid of issues or challenges! They eat them up for breakfast.

You’ve already determined that your retention problem is due to an onboarding problem, so you move fast and throw some things at the wall to see if they stick.

• More explanatory text!

• Less text!

• Force the user to do the thing that shows value!

• Tooltips!

• Dedicated customer support managers!

• More tooltips!

Some of your attempts bomb. Some of them seem to work for some people but not enough. Some of them work really well but won’t scale. Sometimes you stumble on a fix that works and mistake luck for knowing your user.

Does any of this sound familiar? This blog post series covers:

1. What retention problems look like
2. Common fixes and their pitfalls
3. Digging down to the root causes
4. Designing the solution
5. Implementing the design

Spoiler alert: The problem and the solution involves understanding your users.

---

Stay tuned for part 1, coming soon.