Navigating Design Impact
A guide to demystifying design metrics and building a toolkit for effective data-driven design leadership
Designers in tech struggle to measure the impact of their work, as it often relies on subjective metrics such as customer sentiment. Additionally, it's common for design function to lack the necessary infrastructure and budget to gather relevant data to reflect its impact. This challenge is not about artists' profiles struggling with business language; it's about understanding how design rigor is needed and how design leadership can enable and advocate for the necessary infrastructures and processes. You might be surprised by how business-savvy artists' profiles can be. So, as a starting point, I consistently challenge the lack of business acumen among designers
Finding Common Ground: A Holistic Approach
UX Design, like architecture, marketing, and car manufacturing, lives in the realm between aesthetics and engineering/data. They must be both desirable and functional, much like UX design. So, how do these disciplines measure impact, and what can UX designers learn from them?
The significant leap that happens when a design expert transitions into leadership is their capacity to explain to all kinds of senior leadership of the value of design investments. While there’s a limit to how much a designer should justify their value, having a toolkit and framework to demonstrate design’s impact on business goals becomes invaluable, especially when driving cultural change.
UX Measurement: Crafting a UX dashboard metrics for success
Organisations usually focus on key business metrics like North Star, global OKRs, etc.
You want to track what your customers say they do and what they actually really do to be able to gauge the real impact of customer behaviour on business.
To measure UX effectively and connect UX metrics with overall business goals, design function requires a solid monitoring system, which many data pipelines neglect as they primarily focus on engineering or sales metrics.
Often, the challenge will be building a robust system for UX metrics. A well-designed UX dashboard is crucial for collecting essential UX data. As a design lead, your role involves leveraging influence to kick-start the development of an infrastructure that automates the impact on business objectives.
And what is the basic UX data that you would like to collect?
Desirability Scores, what your users feel:
Indicators of customer satisfaction or the likelihood of recommending (NPS, CSAT).
Evaluative UX Metrics, what your users do:
Usability Metrics such as Task Success Rate, Time on Task, Error Rate...
Retention Metrics: User Churn Rate, User Retention Rate, Repeat Visits..
Conversion Metrics: Conversion Rate, Abandonment Rate, Funnel Drop-off Rate…
Infrastructure UX Metrics, what your users ‘suffer’:
Accessibility Metrics: WCAG Compliance, ARIA Compliance
Performance Metrics: Page Load Time, Response Time…
Content Metrics: Content Relevance, Readability…
👉 Tip: You can start by collecting several of all the data points, step by step, like Pokémon, until you collect them all.
Once you can collect relevant UX metrics at scale, you’ll be able to connect these UX data points with the global business goals and establish a cause-and-effect relationship between design decisions and business outcomes.
However, tracing causality effects is not a task for the faint-hearted. You’ll require the invaluable support of data insight teams.
At some companies, data insights and design report to the same leadership and work closely together and if this isn’t the case in your organisation, you’ll need to establish a connection with these data experts, as they are crucial stakeholders when implementing a design data-driven approach.
The Design Impact Metrics Toolkit
Once the data experts and you become BFFs, let’s dive into what I call the Design Impact Metric Toolkit. This toolkit will provide your organisation with a clear sign at a first sight of whether your design health is heading in the right direction or not.
1. UX Debt Index
The concept of UX debt is intuitive: just like financial debt, it results from taking shortcuts or not paying enough attention to important details. The basic idea is that generating any kind of design artefact doesn’t create additional debt, so producing any type of design output costs less than maintaining old legacy systems.
Examples of UX debt can include accessibility issues or half-baked concepts, among others. The math is straightforward: the less UX debt you have, the better it is for your company. However, make sure that you don’t become obsessed with achieving a 100% UX debt-free system—aim for something around 75/25, as it will be the most efficient in terms of cost and effort.
2. Design Heuristics index
Design heuristics are a set of guidelines that can help you measure the usability and effectiveness of your design. They are not strict rules but a set of principles that can guide your decision-making process and allows you and your team to don’t have to reinvent the wheel every time, like short-cut they allow you to establish a foundation that is likely to work well. It will spare a lot of hours of A/B testing and trial/error attempts. The more you adhere to heuristic, the better your experience would work hypothetically. However, measure them can be tricky and it falls more on the side of team processes. A simple checklist of “to dos” can do the trick to allow heuristics to be measured, but it requires team coaching and committing to specific ways of working.
Examples of design heuristics include consistency, simplicity, and clarity. By incorporating design heuristics into your work. Three are templates that include compressive design heuristic standards that you can use. You can measure the impact of your UX design and ensure that it meets user needs and expectations.
3. User Sentiment Score
User research is an essential tool for measuring UX impact, especially on the qualitative side. By gathering feedback from your users, you can gain insights into their needs and preferences. This information can help you identify areas for improvement and guide your decision-making process. User research can take many forms, such as surveys, interviews, or usability testing. It is crucial to choose the right method based on your project’s needs and goals.
Examples of user sentiment scoring include Net Promoter Score (NPS), Customer Satisfaction (CSAT) surveys, and sentiment analysis of user-generated content. Some modern tools allow you to scan the internet and spot trends in customer satisfaction opinions before issues escalate. Keep in mind the need for a dedicated infrastructure, which is usually a paid service and thus requires a budget, to gather and process these types of insights.
4. Evaluative UX metrics
Integrating these behavioural UX metrics into data pipelines is often the most challenging and time or budget-consuming task. However, once we set them up, they offer incredible value by providing a valuable perspective on user behavior and interaction without requiring direct user feedback. Once you automate these quantitative metrics and establish a strong connection to causality, they can offer invaluable insights into the effectiveness of your design.
Examples of evaluative UX metrics include completion rates, time on task, error rates, heat maps, and eye tracking. These metrics empower UX designers and researchers to make informed decisions and optimise user experiences without the need for direct feedback, based on empirical data and observations. They usually require third-party platforms or tools and a specific budget.
In summary
Measuring the impact of design in corporate organizations is a significant challenge. It requires design leadership to establish the right infrastructure, processes, and budget to engage engineering and data teams and secure approval for third-party tools, all while aligning with business goals.
Creating a UX dashboard with key metrics, such as user sentiment and behavioral data, is essential for connecting design decisions to business outcomes that can substantiate real design impact, and this can take time and several iterations. Collaboration with data insight teams is crucial for this process.
The Design Impact Metrics Toolkit, encompassing the UX Debt Index, Design Heuristics Index, User Sentiment Score, and Evaluative UX Metrics, serves as a practical guide to assess the health of design initiatives. By using these metrics and collaborating with data experts, organisations can show the value of design in achieving business success.
I hope you find this guide useful!
