Onboarding Part 1 – Retention Problems

This is part 1 of my series on user retention and onboarding.

Defining Terms

Let’s define a few terms.

Customer or user retention means keeping customers or users. What does it mean to “keep” a customer or a user? It means that a person keeps using your product or service over time. However, how you define and measure that depends on the business. I touch on some measurements next.

I may use “customer” and “user” interchangeably here. For many products, they are the same but for B2B products, they are different since a customer may easily have multiple users. The retention of a customer is clearly different than the retention of one of their employees who uses your product (though they are likely correlated).

Onboarding refers to the period starting when a customer or user begins using your product until the point they are able to get some value from it. For complex products, onboarding could be a multi-week process that includes training classes. For simple products, it could be instantaneous.

SaaS stands for Software as a Service and describes both technology and business model. SaaS products are delivered remotely over the internet (i.e., there’s always some component running in the cloud) and are sold on a subscription basis. I am primarily writing about this type of product.

Measuring Retention

How do you measure retention and what do user retention problems look like? Ultimately, they are revenue problems – the company isn’t meeting financial targets. This blog series isn’t about product management, sales, or marketing, so let’s skip a bunch of analysis and assume that the marketing funnel is delivering the right number of prospects, you’re charging the right price, and you think you have product-market fit… if only users would just stick around.

In short, you’re getting enough new users but they aren’t coming back for some reason.

Depending on your revenue model, you may not see a problem right away. For example, if you sell annual subscriptions, it could take a year to realize customers are not renewing at the expected rate. So, most companies identify metrics that may indicate retention problems well before the renewal comes around. Some common ones:

  • Active users
  • Frequency of key actions (e.g., logging in, running reports, executing automations, posting content)
  • Customer support calls
  • Response rate to outreach

Retention is such an important business metric that you can be sure someone outside of the user experience team will notice problems. When that happens, how might the UX team be called on to help?

Calling User Experience

UX is an obvious choice to bring in to fix retention problems. For a strong UX team, I would expect they tackle the problem like any other starting with research and followed by an iterative design process with lots of user feedback to solve it.

However, there are still many companies that don’t operate this way and attempt to fix the problem based on assumptions and quick fixes. In this case, the UX team may be prescribed work. There are a couple interesting contexts: low retention of paying customers and low retention of trial or freemium users. Let’s look at some of the things a designer may be asked to do. 

Losing Subscribers

Almost every product uses a subscription model these days. When retention of paying customers is a challenge, the UX team may be tasked with things like:

  • Improving the usability of key functionality because low usage means it’s too hard to use
  • Adding more help or documentation or pointers to these features
  • Designing more new features to fill an assumed gap

In these situations, it’s sometimes a little clearer to everyone what the problems are since it’s easier to talk to customers than to semi-anonymous people just trying out your product. These fixes also assume that paying customers have gotten through the onboarding stage successfully to some extent. If not, requested fixes will sound a lot like the ones below.

Free Trials or Freemium Products

Many companies offer free trials or a free tier hoping people will sign up and the product will sell itself. These are challenging models to get right and to diagnose the problems when things go wrong!

Trials have their own retention metrics and, as a bonus, you can get feedback on the order of days or weeks. When trial users leave and don’t come back after their initial session or two, UX may be asked to tackle things like:

  • Explaining the product’s value proposition when users first sign up
  • Walking new users through the product through wizards or tooltips
  • Doing more automatic setup or configuration
  • Adding video explainers because people just don’t read

I like to summarize these requests as fixing the onboarding.


In part 2, I take a step back and consider some real-life examples and see just how these issues cross organizational divisions and how deep they can go.

Balancing Acts: Too Little vs. Too Much

pexels-photo-235990My previous post was about balancing problem definition against design and implementation. The balancing act doesn’t end there, though. Even after defining the problem, there are usually multiple solutions to choose from and each of those can be considered complete with varying levels of functionality.

The goal is to deliver a well-balanced solution that offers just the right amount of functionality. There are easily dozens of decisions about the design and implementation that can throw the balance off. I’ve seen and fallen into a few patterns where these design and implementation tradeoffs didn’t quite balance.

Too Much Left on the Punch List

Some teams (engineering, design, or product) push back on basic functionality to cut down on initial effort. For example, the design requires the user can create an object but, to save a few hours of work, we’ll defer implementing rename, delete, or edit until later. They argue that these are easy to add later and the engineering team can always make these changes manually, especially early on when there are few customers.

No matter your role, it’s tempting to accept this argument. As a product manager, you can fit in an additional feature. As a designer, you are freed up from considering some design details. As an engineer, less code means fewer tests and fewer bugs. (OK, maybe it just means different bugs. 😉 )

Don’t fall for the argument. Returning to “completed” features later after the product has grown is harder than it seems. Eventually, what would have been fairly quick to add before becomes difficult as the team loses familiarity with that area of the code or, worse, changes are made only to uncover surprising dependencies and new bugs (“What? That name could never change before!”).

Omitting such basic functionality, even if you expect people will rarely need it, can lead to your users:

  • feeling they have an unfinished product
  • frustrated by attempting to find something that’s not there
  • frustrated by the need to contact support
  • abandoning that functionality or the entire product

Teams often cut corners when they doubt that the functionality is here to stay. Perhaps the product vision is unclear or there has been a recent pivot that undercut trust in the vision. In that case, you either need to convince the team that this work is going to stick around or you need to do the researching, testing, or experimentation needed to convince everyone (including yourself) you’re on the right track.

Gold Plating

It’s easy to overcompensate in the other direction and build unnecessary or low-value functionality. Obviously, doing unnecessary work consumes development time better used elsewhere. Gold plating also risks:

  • complicating the user experience
  • additional documentation or training effort
  • creating a maintenance burden
  • additional customer support

Finding the Balance

casey-horner-460825.jpgThe performer finds balance through feedback, practice, and experience. Designers achieve balance the same way. The tightrope walker knows when they’ve fallen off the tight rope but it’s harder to know when a design is off balance.

Trying out design ideas with users is critical to get early feedback. As they use the design prototype to complete realistic tasks, are they using all the functionality? Are they missing functionality? What can be removed? What must be added?

As useful as usability testing is, it can’t replicate reality. It’s important to monitor how the design performs in real life. Luckily, we have a many tools at our disposal:

  • Usage analytics tools like Heap and Pendo
  • Customer support feedback
  • User research – interviews, observation, surveys, etc

How much time you put into collecting and analyzing data depends on the size and importance of the feature. However, in order to improve your experience, you need to do something. Learning from each balancing attempt will help you get the balance right earlier in the process and spend more time finding new problems to solve.

 

Balancing Acts: Thinking vs. Doing

barnum_and_bailey_grand_water_circusDoes your work ever feel like a circus? Yeah, mine too. I feel like I am always performing. My most popular performance is the balancing act – it turns out that product design is a series of balancing acts.

As designers and product managers, we are always balancing the good-cheap-fast triangle. No time to fully consider a design? Whiteboard it and check off “cheap” and “fast”. The lone UX designer with a bit of lead time? Check off “good” and “cheap”. Sometimes, the balance is determined by outside forces. Other times, we have more influence.

Let’s take a look at one of the most popular balancing acts that I call “Thinking vs. Doing”.

Thinking vs. Doing

How much time should you spend understanding the problem you need to solve and finding the best solution before starting implementation? You can ask this question when embarking on a new product design or when adding a small enhancement. Of course, the larger the project, the more important it is to understand the problem before getting too far with design. Larry Marine likes to say you should “avoid solving the wrong problem very well.” He’s absolutely correct.

Now, I use “thinking” facetiously. Understanding the problem requires more than thinking. It’s research and research is work. From the outside, though, it may look a lot like thinking, so you may be tempted to cut corners at this early stage. For example, I know that an idle engineering team can result in pressure to speed things up.

Here are some tips to do the right thing while keeping things moving and knowing when you are done.

Start with Assumptions

List your and your team’s assumptions about the product/feature as soon as you can. Involving the wider team will ensure you generate a comprehensive list. These assumptions include the market, users, proposed functionality, and technology. You can capture these as simple lists, provisional personas, requirements, business plans, or whatever combination works for you.

Next, for each assumption, determine your confidence level. Items with high confidence should still be validated, but these are also areas where you can press forward with design and implementation right from he start.

Involve the Entire Team

Is that idle team pressuring you? Get them involved in the research. At all times, it’s best to have engineers tag along on research activities and help analyze results. But, if the team is truly waiting on the research, you could quickly train a few people (who have the proper aptitude) to do some independent research activities.

Enough is Enough

Do enough research to validate or invalidate these assumptions and better define the problem space. The best guideline I’ve heard is to research until you are no longer surprised. As soon as commonalities start appearing, you should be able to firm up some of the initial assumptions and commit to bigger decisions.

You should also continue research activities throughout the life of the product (for example, regularly usability testing), so it’s OK to shift focus to solutions as soon as you can.

Solving the Problem

Once you think you’ve got the problem defined, it’s time to design the solution. This is another activity that needs careful balancing as you could easily search a long time before you find the “best” solution. At this point, I like to bias the effort towards finding a “good enough” design quickly.

Determining “good enough” is a blog post (or book) of its own. However, your problem definition will help determine what that means in the product’s context.

Keep Moving

It’s important for the entire team to continuously make progress. As a designer or researcher, one of your responsibilities is to communicate with the team. This communication needs to include more than deliverables – it needs to include the entire process so everyone shares a sense of progress.

Finally, after doing such a good job at balancing the problem discovery work, never lose sight of the problem you are solving. The act of design and feedback often leads to new insights into the problem and it’s important to incorporate those and adjust as you go.

My users! No, MY users!

One set of design challenges when working on enterprise products is the number of different people and roles involved. Many designers and product managers are familiar with the buyer vs. user split where the people using the product aren’t the same as those making the purchase decision.

Product teams have historically given more weight to the buyers and their opinions, requirements, etc. This is for good reason – if the buyer doesn’t buy, your product goes nowhere. Catering exclusively to the buyer usually results in a lousy product that sits on the shelf or is despised. With ever more SaaS products that are sold on a subscription basis, ignoring the end user will result in poor adoption and renewal rates.

Anyone practicing user-centered design knows the importance of studying and working with the people doing the actual work. So, why is it so hard to get access to these people?

While I was working on an enterprise file sharing product, I realized it wasn’t just a simple case of finding the right person to interview. Many of our contacts were IT staff either administrating or evaluating our product. I had many good meetings with these folks, but my frequent requests to meet with our product’s end users often went nowhere.

One day, an IT contact’s comment finally provided the insight to why I wasn’t getting past IT. IT departments are responsible for evaluating, purchasing, and maintaining their company’s systems. So, IT is also responsible for understanding their company’s employees’ needs. Asking to interview the users was either being interpreted as redundant or, worse, insulting!

Getting through these gatekeepers is challenging and requires persuasion and persistence. There are a few approaches to try:

  • Explain the importance of getting access to end users and how your research methods and first hand data are crucial to providing the best product for their money.
  • Treat the IT staff as surrogate users. Conduct interviews, usability tests, and other research and design activities  with them. However, push the boundaries of their knowledge and look for the opportunity to get access to actual users to answer outstanding questions.
  • Recruit representative users from outside the company. You’ll avoid possibly wasting time but may not discover needs and design solutions to help your actual customers.

Share your experiences in the comments.