When it comes to user research and product development, bias can have big implications. 

Bias in user research refers to errors that can influence the results and conclusions of a user study.

In these instances, bias can lead to a skewed understanding of your user behaviour, preferences, and needs, ultimately affecting the quality of the insights you gain from the research, and creating misguided design decisions in the long run. 

As Zoe Glas says in a podcast from User Interviews, “We aren't impartial. We never are. [But] we can do our best to be fair.” And we can be fair by identifying and mitigating bias as much as possible to make sure that our findings are accurate and representative.

‍

Types of bias

First up, let’s take a look at the different types of bias that can impact user research.

‍

Sampling bias

When doing user research, if the participants are not representative of the actual end-user base, it can lead to sampling bias. 

Product decisions based on this data won’t align with the user base, and can lead to products that don't meet user expectations.

‍

Confirmation bias

Confirmation bias happens when researchers or product teams look for information that confirms their pre-existing beliefs or hypotheses.

This can lead to ignoring contradictory data and making decisions based on incomplete or skewed information. 

‍

Cultural bias

Some researchers may unintentionally favour their own cultural norms and values, leading to a lack of understanding or appreciation for the perspectives and needs of users from different cultural backgrounds.

This can result in making products that are less inclusive or relevant to a global user base.

‍

Observer bias

Observer bias happens when the presence of researchers or product team members influences participant behaviour or feedback.

Users can change their behaviour or responses because they feel watched, which leads to inaccurate insights. 

‍

Data bias

Data used in product development, such as historical user data or machine learning training data, can contain biases if it reflects past discriminatory practices or reinforces stereotypes. 

The main impact of data bias is that it can lead to products that inadvertently discriminate against or harm certain users.

‍

Accessibility bias

If user research methods and product designs don’t include the needs of users with disabilities or specific needs, it can lead to accessibility bias.

This can result in products that are not inclusive or usable by individuals with different abilities.

Products built on biased foundations often end up unfairly excluding, stereotyping, or harming users. The later in the process this bias is identified, the more difficult and costly it becomes to fix. 

That’s why building in checks and balances to uncover bias from the earliest phases of product development is key. 

Preventing bias 

By actively addressing bias, researchers can achieve a more comprehensive and unbiased understanding of user behaviour and expectations. 

This helps to create products and experiences that genuinely cater to users, resulting in increased user satisfaction, better market adoption, and ultimately, improved success. 

Mitigating bias isn’t just a best practice; it's a commitment to providing equal and inclusive solutions for all users.

But what can you do to mitigate bias in user research? Here are the tools that we at 383 use to ensure we’re getting the fairest research results. 

‍

Selection process

For most of our research projects, we conduct customer interviews and work closely with our clients to guarantee the inclusion of a diverse and representative participant pool. 

We consistently encourage our clients to explore their customer segments, allowing us to get a strong understanding of the demographics they target, making sure that we engage with a representative spectrum of participants. 

When our clients take charge of the initial participant recruitment, we provide them with a recruitment pack. This pack includes guidelines on considering underrepresented groups during the recruitment process. Diversity is very specific to each organisation, so we provide personalised guidance for each organisation. 

Once our clients provide us with a list of individuals interested in participating, we use a  meticulous selection process to choose a diverse range of people within each segment. This process aims to ensure diversity based on factors relevant to our client's business. 

When we recruit participants for interviews, we also take proactive measures to make sure underrepresented groups feel at ease and confident about participating. We think it’s really important to address concerns related to the format for research participation. 

While our internal preference is conducting research through video interviews, we know that some individuals may not feel comfortable with this format. Therefore, we offer alternative channels to accommodate diverse participant preferences.

‍

Jobs to be done

We place a strong emphasis on qualitative research. We know that meaningful user dialogue is the key to crafting services that align perfectly with their needs.

During user research, we use the Jobs to be Done framework (JTBD), which involves conducting in-depth interviews and observing customers in their everyday settings. 

This qualitative research method helps us prioritise hearing and understanding customers' genuine experiences and obstacles, instead of seeking validation for preconceived notions. 

Our approach involves asking open-ended questions and conducting conversational interviews, allowing us to tailor our questions based on the insights shared by the interviewees.

JTBD also helps us to address confirmation bias because it’s a descriptive framework, not a prescriptive one. It doesn't dictate what the solution, ideas or initiatives should be, but helps us to understand what customers are trying to achieve. This approach reduces the tendency to confirm existing beliefs about the ‘right’ solution.

‍

User segmentation

One of our primary strategies for mitigating bias is our user segmentation approach. This involves collecting data per user segment, allowing us to uncover subtleties in customer needs and preferences. This approach helps prevent bias by avoiding a one-size-fits-all assumption.

As part of our process, we conduct research synthesis only after collecting all data to prevent premature assumptions based on initial interviews. 

We try to avoid giving early previews of feedback. Although it’s tempting to want to see findings as soon as possible, it’s important to wait and gather all the data before making any initial observations and assumptions.

‍

Research team

The research team involved in both collecting and analysing data is typically conducted by a team of two individuals, each with distinct roles -for example, a Senior Product Manager and a Senior Strategist, or a Senior Strategist and a Senior Designer.

It’s really important for us that we have this pairing, because we often come from diverse backgrounds, have different experiences, and can offer a variety of perspectives.

The diversity within the team encourages the identification and mitigation of each other's biases and viewpoints. And for those who take part in the research, we've provided training to help them recognise potential biases in our research processes.

Limiting bias  

We know you can't avoid bias entirely, but you can identify and reduce bias in UX research. Here are some of our top tips for keeping bias to a minimum.

‍

Diverse participants

Make sure your research participants represent a diverse range of demographics, including age, gender, race, socioeconomic background, and other relevant characteristics. 

‍

Random sampling

Random sampling techniques minimise the chance of inadvertently introducing bias by only selecting participants who are easily accessible or convenient.

‍

Inclusive recruitment

Avoid relying only on your existing network or specific channels that can skew the sample. Consider using multiple channels to reach a broader audience.

‍

Informed consent

Clearly explain the research objectives and the participant's role in the study. Make sure participants understand the purpose and potential outcomes of the research. Get informed consent before starting the study.

‍

Neutral wording

Make sure any survey questions, interview scripts, and communication with participants uses neutral language and avoid leading or biased phrasing that could influence responses.

‍

Balanced moderation

If you're conducting interviews or usability tests, be a neutral moderator. Avoid leading questions, and keep a neutral tone during the research process.

‍

Blinding

Consider blinding the research team to certain participant characteristics, such as gender or age, during data analysis.

‍

Triangulation

Use multiple data collection methods, such as surveys, interviews, and behavioural observation, to cross-validate findings.

‍

Bias awareness training

Train your research team to be aware of potential biases and how they can influence the research process. Encourage them to check their own biases and maintain objectivity.

‍

Review and reflect

Periodically review your research practices and reflect on potential sources of bias. Make adjustments as needed to improve the research process continually.

‍

By implementing these strategies, you can minimise bias in your user research and ensure your findings are reliable and representative of the diverse perspectives of your user base. 

Completely eliminating bias is challenging, but by being aware of potential sources of bias and actively working to address them, you can conduct more objective and valuable user research.