A strategy for defining and tackling UX debt

💡 Note: This article was published originally on athenahealth’s design blog in August 2018. Since then, the UX org at athenahealth has maintained this system, and even created OKRs around reducing this debt in the product.

athenahealth provides network-enabled services for healthcare and point-of-care mobile apps to drive clinical and financial results for its hospital and ambulatory clients — we employ 85+ designers globally to support those efforts. Collector, one of our core products, helps clients improve their financial performance by helping them reduce the heavy administrative costs associated with getting paid.

In the early days of our Collector product, athenahealth was a startup environment with no dedicated user experience team. As with so many startups, there was tremendous pressure to quickly deliver functionality to meet the needs of a rapidly growing and evolving client base. Often, "good enough" was the right business decision. As athena's oldest product, Collector has accumulated its share of design debt over the years. Users have had to live with misaligned layouts, unclear copy, and clunky workflows. Individual issues, taken on their face, can seem trivial and not a priority to fix. Taken in aggregate, these issues can lead to frustrated customers, inefficient workflows, and eroding trust in the product.

Flash forward to today. athenahealth is a mature company with a firm commitment to strategic product design. This gives us an opportunity to start paying down some of the design debt we've taken on and improve our users' experience with the product.

First, let’s talk about what UX debt is - and is not.

On our team, we define “UX debt” as any design issue that impacts the accuracy, efficiency or discoverability of a user’s work, or which impacts the credibility of our product to our users. It may come from inconsistent interaction or visual design, inadequate system messaging, or excessive cognitive load required to perform certain tasks.

To quote an article on the issue from UXPA Magazine:

The term “technical debt” was coined by Ward Cunningham as a metaphor used to describe the increased cost of maintaining technology due to shortcuts taken during the development process. Joshua Kerievsky later extended the metaphor to user experience design using the term “User Experience (UX) Debt.” Kerievsky explained that, like technical debt, UX debt will eventually come due, usually in the form of less customer satisfaction and possible customer defects.

That said, all UX debt is not created equal. The universe of design problems can range from seemingly trivial fixes (fix some spacing, improve some system messaging) to entire workflows that require redesign. In most conversations, teams will take a one-or-the-other approach; UX debt is either a minor fix to specific UI elements, or it’s the need to redesign multiple entire workflows. This typical framing of UX debt creates problems with prioritization. The problem is either too small/trivial to prioritize, or it’s so big that the team doesn’t want to risk working on it.

With this in mind, we needed a structured way to:

• define the types and scopes of debt we were dealing with;
• store and prioritize the backlog of debt alongside the rest of the product roadmap;
• determine ways to track the “payment” of debt across sprints and releases;
• focus conversations around constructive, manageable things we could fix, while also taking note of where larger redesigns were more appropriate.

In other words, if the team is going to commit to paying down UX debt, the debt first needs to be clearly defined, catalogued, and prioritized - not bundled into an amorphous group of UX issues.

Identifying UX debt by problem scope

Through discussions with the team, we organized our definitions around two fundamental characteristics: scope (how hard/easy it is to fix) and locus (global vs. local). Each of these types of debt can then be treated in different ways, making it easier for teams to prioritize the work against the larger product backlog. These types of debt are listed below:

Quick win

What it is: A UX issue that is relatively small in scope, with a clear fix that can be implemented by a developer and designer working together in a single sprint with minimal artifacts or upfront design required. Examples of this type of debt could include updating error messages or instructional copy, or making minor layout tweaks, such as spacing or a color definition.

What we do about it: combine as many issues as possible that impact the same screen into a JIRA ticket that can be prioritized in the backlog.

Optimization

What it is: A UX issue that is local in nature (impacts a single element or a piece of a single screen), has a clearly understood solution that will require a non-trivial amount of work by design or development to implement, but can be resolved within a sprint or two. Examples of this type of debt could include updating an old UI pattern to use a pattern from our Forge Design System, or changing a field of checkboxes to use a multi-select list.

What we do about it: Clearly define the current state and the expected state, and add as a JIRA ticket that can be prioritized in the backlog.

Redesign effort

What it is: The business problem is known and we have a functional but sub-optimal solution in place. The workflow/feature as designed has multiple issues across several distinct screens, and the most appropriate solution is not necessarily obvious. Improving the workflow will require significant research and design effort, including usability testing. Examples of this could include redesigning the way that users turn chart information into billable codes, or improving the way that chart information is displayed to coding users.

What we do about it: Since these issues are bigger than a single JIRA ticket, and the solution isn’t immediately apparent, they are prioritized as design initiatives within a separate backlog.

Our approach to tackling the debt

Once we’d built a shared language around our types of debt, it was time to start finding and cataloguing it for prioritization. Taking a cue from Catriona Shedd’s structure for defining design debt, we started by identifying the high-level user goals and tasks. We then used Capian to conduct an expert review of each workflow. In addition to the expert reviews, we started to sift through our existing user research archives, design scoring feedback, and notes from regular user feedback calls to start finding potential issues to address.

Then came the question of storing and tracking it all. Spreadsheets were easy to lose track of, or get cluttered with too much information. Adding every single issue to JIRA was a sure-fire way to torture the development teams. We ultimately decided on an approach that involves two key streams:

• JIRA to store and track issues that are well defined and ready to be worked on by the development team;
• Trello to capture high-level issues that still need to be shaped, discussed, etc.

This allows the development backlogs to stay focused on issues that are workable, while the UX team can keep track of the other issues, pulling them into the development queue as they’re shaped. We use labels within our Trello cards and JIRA issues to keep track of the various types of debt, and can run JIRA reports on those labels to see how we’re tracking against paying it down.

Figuring it out along the way

By undertaking this effort, we hope to achieve the following:

• An inventory of design debt within Collector that we can track and drive down over time. To paraphrase Peter Drucker, "You can't manage what you haven't measured."
• A robust process for identifying, cataloging, and addressing design debt going forward.
• Visible improvements to the quality of the user experience for our clients.

As we take on this effort, we’re actively taking input from our Product and Engineering colleagues about how it's going and what we can do to make it as effective as possible.