Learn more about research methods

Understanding your users is one of the most essential parts of UX design. You’ve already learned about the problems that can come up when designers make assumptions about their users and their needs. So how do you avoid making those assumptions? By doing research! In this reading, you’ll explore some common research methods used by UX designers.

Types of research

There are two ways to categorize research: who conducts the research and the type of data collected.

The first way to categorize research is based on who conducts the research: primary research and secondary research.

Primary research is research you conduct yourself. Information from direct interactions with users, like interviews, surveys, or usability studies, are considered primary research.

Secondary research is research that uses information someone else has put together. For example, using information from sources like books, articles, or journals is considered secondary research.

The second way to categorize research is based on the type of data collected: qualitative or quantitative.

Qualitative research is primarily collected through observations and conversations. Qualitative research is based on understanding users’ needs and aims to answer questions like “why” or “how did this happen?”

Quantitative research focuses on data that can be gathered by counting or measuring. Quantitative research is based on numerical data that’s often collected from large-scale surveys. This type of research aims to answer questions like “how many?” and “how much?”

All four of these types of research can intermix. Primary and secondary research can be both qualitative and quantitative. For example, an interview is qualitative research. An interview conducted by you is primary research. If you review an article about an interview conducted by someone else, it’s secondary research. It’s important to be able to identify the difference between these types of research because the data you collect forms the basis of your design decisions.

Primary research methods

Now that you understand the different types of research, let’s review some common primary research methods for gathering information.

Interviews are a research method used to collect in-depth information on people’s opinions, thoughts, experiences, and feelings. Interviews can be performed one-on-one or in a group setting, like a focus group.

Interviews can take the form of qualitative and quantitative research. A qualitative research method includes open-ended questions that require participants to explain their answers by providing more details. A quantitative research method includes only close-ended questions, like questions that require only “yes” or “no” responses or set multiple choice questions.

Best practice is to conduct at least five user interviews during your research. As you conduct your interviews, you’ll start to find similarities in the feedback that users provide about what works and what doesn’t work about your product. This is exactly the kind of feedback you want!

Advantages

• You’re better able to understand what a user thinks and why.
• You can adjust your questions or refocus the discussion based on the user’s answers.
• You have the ability to ask follow-up questions in real time.
• You have the ability to ask questions specific to a user’s needs.
• You’ll receive direct suggestions from the user.

Disadvantages

• It’s time-consuming to interview each user.
• It’s expensive to pay participants and to rent space for the interviews.
• The sample sizes are smaller, due to time and money constraints.
• Group interviews can be affected by the bandwagon effect, or going along with the group’s opinion instead of thinking creatively, which can discourage open discussion by people who have an opinion that doesn’t align with the majority of the group.

If you want to learn more about interviews, check out an article about user interviews from the Nielsen Norman Group.

A survey is an activity where many people are asked the same questions in order to understand what most people think about a product. Surveys are a great way to measure the success of your product, during development and after it’s launched. For example, sending surveys after a product is released can help you measure the effectiveness of your product and provide a foundation for future improvements.

You can design surveys to include open-ended questions for qualitative research, which allow research participants to clarify their survey responses, as well as close-ended questions for quantitative research, which generate numerical data.

Advantages

• You can learn more from a larger sample size.
• You are able to gather results and insights quickly.
• Surveys are usually inexpensive because they don’t take as much time for participants to complete, and they can be done remotely.

Disadvantages

• Surveys often do not allow for in-depth feedback; most questions will have responses drawn from a set of multiple-choice answers.
• There are some types of research questions that won’t work in a survey format.
• Surveys usually do not allow for personalization.

If you want to learn more about surveys, check out usability.gov’s article about online surveys.

A usability study is a technique used to evaluate a product by testing it on users. Usability studies help demonstrate if a product is on the right track or if the design needs to be adjusted. There are lots of ways to test usability, both in person and online. It’s a good idea to record your usability sessions, either audio or video, so you can reference the user data as you make design decisions later on in the process.

Qualitative research is based on observations, and a critical part of conducting usability studies is observing how participants interact with the product you’ve designed. Focusing on qualitative research during usability studies can generate more personal insights by assessing the behavior of users as they experience the product. Quantitative research can also be used when conducting usability studies to understand participants’ impressions of the product.

Advantages

• You can learn from first-hand user interaction and observation.
• Usability studies can challenge your assumptions about your product by demonstrating a completely different result than you were expecting.
• Users can provide in-depth feedback.

Disadvantages

• Usability studies only measures how easy it is to use a product.
• This type of research can be expensive, especially if it’s conducted in person.
• There can be differences between a “controlled” usability study in a lab versus how a user experiences the product in their real life.

If you want to learn more about usability tests, check out the Nielsen Norman Group’s article on usability testing.

Secondary research methods

Secondary research can be completed at any phase of the project, since you’re using information from outside sources. In other words, secondary research is not a direct result of your product or the user you’re designing for. The information you discover during secondary research might lay a foundation for your primary research, so you have a better idea of where to focus your efforts. Or, secondary research might supplement the findings from your primary research for a project, to reiterate or strengthen your conclusions.

Advantages

• Secondary research is generally cheaper and faster than primary research. This means you’ll save time and money.
• You can often find secondary research via online searches and subscription research publications.
• Secondary research can be a good supplement to findings from your primary research.

Disadvantages

• You will not learn from any first-hand user interaction.
• You will not receive user feedback specific to your product.
• Secondary research can be misleading and generalizing if not done appropriately.

If you want to learn more about secondary research, check out an article about secondary research from Formplus.

Research in practice

As you move forward in your UX design career, you’ll likely get to experience using several different types of research methods. Knowing the advantages and disadvantages of each method, and when to use each, can make your research more effective and can improve your product’s design.

If you want to explore research further, check out this user-experience research methods article from NN Group. It will guide you through choosing the best research method for you out of 20 popular options.

The human brain is an incredible processing machine, and it can store an amazing amount of information. One way brains are able to store so much information is by creating mental shortcuts based on repeated patterns. These shortcuts allow humans to relate and group information together for quicker processing. But, these repeated patterns of thinking can lead to inaccurate or unreasonable conclusions that are biased — favoring or having prejudice against someone or something. Biases can seriously impact your user research and negatively influence the design of your final product. So, let’s explore how bias can affect your work as a UX designer and how to combat it during your research.

Preventing bias in data collection

It’s important to note that everyone has biases. It’s just a natural part of being human. Being able to recognize your own biases and prevent them from affecting your work is what really matters. As a UX designer, you’ll need to know how to anticipate, identify, and overcome biases in your research, in particular.

Choose your words carefully. While conducting research, it’s important to use words that don’t lead the user in one direction or another. Of course, as a designer, you’re going to be partial to the designs you’ve created, and you’ll likely assume that users will appreciate them too. That’s why you designed them! But when asking users questions about their experience using your product, you don’t want them to answer in a particular way just to please you. Choosing leading words can cause the framing effect, where users make a decision or choice based on the way information was presented to them.

This is especially critical in usability studies. For example, imagine a participant is testing your designs. You ask the participant: “Do you like or dislike the improved layout of these buttons?” Because you used the word “improved,” the user will most likely reply positively. But, this isn’t very useful feedback because you framed the question in a way that led the participant to respond accordingly. To improve your product, you need honest feedback.

Instead, a better way to frame the same question is: “Explain how you feel about the layout of the buttons.” This phrasing allows the user to come to their own conclusions without any outside influence, which will give you better data about their thought process and experience.

Foster independent thinking. Group interviews can be affected by the bandwagon effect, or going along with the group’s opinion instead of thinking creatively, which can discourage open discussion by people who have an opinion that doesn’t align with the majority of the group.

For example, imagine you’re conducting research with a group of five participants. You ask each person in the group to share their thoughts one at a time about a particular product design choice, like the placement of a button on the home page. By the time the last person shares their thoughts, their feedback will be affected by all of the answers that were shared before them. To combat the bandwagon effect, ask participants to write down or record their thoughts before discussing as a group.

Avoid specific language. It’s important to be mindful about the types of questions you ask users and how those questions are framed. You’ll need to be careful to avoid confirmation bias, which is trying to find evidence to prove a hypothesis you already have.

Confirmation bias is particularly prevalent in online surveys. For example, imagine that you’re conducting an online survey with a large group of participants. One of your survey questions is: “How do you use our product?” As the designer, you have a few ideas about how you think people use your product, so you provide four options with specifically worded language that the participant has to choose from. If none of the options you’ve provided apply to the user, they can’t select “other” or skip the question, so they’ll be forced to choose one of the multiple-choice answers that doesn’t match their actual experience. That means you’ll end up with false information that skews your research data and potentially provides incorrect evidence for a hypothesis you already had.

Remember, in a survey, you want measurable results, which is known as quantitative data. You can reframe the question in your survey to ask participants to rate their experiences using the product, which will be a more accurate way to gauge how they felt about using it.

Limit the guidance you give users. Everyone learns and thinks in different ways. When you’re conducting any type of UX research, you have to be cautious to avoid experiencing any false consensus, which is the assumption that others will think the same way as you do.

If you’re conducting a usability study, some of the participants will not follow the product’s user flow in the way that you might expect. For example, a user might click through the menu, select a folder, and then select a subfolder to complete a task you assigned them, when there’s actually a simple hyperlink on the homepage that could have saved them time. In addition, some participants may use assistive technology to navigate the product and might follow an entirely different flow.

It’s important to let participants follow their own paths through your product, without interrupting them. Interrupting a participant while they’re experiencing your product will deprive you of useful data that can help you understand how to improve your designs. Instead, ask participants to narrate or break down their user journey with your product, as they move through the flow. This will allow you to better understand their thought process as they navigate through your designs.

Consider users’ tone and body language. You’ll work with many different users and participants throughout your UX career, and part of your job will involve interpreting their nonverbal cues, like vocal tone and body language. To avoid experiencing implicit biases, which are based on the collection of attitudes and stereotypes you associate with people without your conscious knowledge, it’s important to clarify when you think you’re getting mixed signals from a participant.

For example, imagine you’re conducting a one-on-one interview, and the participant has their arms crossed over their chest. This can be interpreted as a sign of feeling defensive or insecure, which might contradict the positive feedback they are sharing verbally about your product. This is a great time to ask the participant questions, like “Is any of this making you uncomfortable?”, which can encourage them to explain that it’s cold in your office and they’re just trying to warm themselves up. Always ask questions if you’re unsure about the intention of a user’s tone or body language!

For this feedback process to work, however, it’s important to make sure participants are comfortable sharing their thoughts with you. Before the research begins, ask participants about themselves or make light conversation. Starting with easier questions can help reduce anxiety or awkwardness throughout the study.

Be careful of your own body language and reactions. You also have to be mindful of your own tone and body language while interacting with participants. Social desirability bias can happen when a participant answers a question based on what they think you want to hear. If you ask a question to a participant, and they notice you exhibiting a visual or audible clue that suggests your own opinion about the question, they might answer in a way that they think will please you.

For example, imagine you’re describing a feature of the app you’ve designed that really excites you, and your tone of voice changes. If this happens, it’s likely that the participant won’t be honest about their negative opinions of the feature, since you’re so positive about it. If you want the data you collect to be useful, the user has to feel comfortable sharing their true, unfiltered feelings about the product. It’s your job to guide them through the process without accidentally influencing their answers. One way to do this is to reassure participants that their answers won’t hurt anyone’s feelings and that you really want to hear their honest opinions in order to improve your work.

Plan your research effectively. Tight deadlines are inevitable. But as a UX designer, it’s essential you get enough time to recruit the right users for your research. Availability bias occurs when you rush the user recruitment process or skip screener questions to attract a bigger pool of users, even if they don’t fit the qualifications or characteristics that you’ve already determined are present in your ideal user.

The research that you collect is vital to your product design process. So interviewing users that don’t fall under your personas won’t give you the data you need to improve your designs. If you’re having trouble recruiting the right users before your deadline, offer a better incentive for participating in your study, adjust your recruitment strategy, or ask your project manager for more time. Don’t just take any user who’s available.

Remain open minded. One more tip: When you’re conducting research, you have to work hard to treat all information equally to avoid both primacy bias, which is remembering the first user more than others, and recency bias, which is most easily remembering the last thing you heard. To help combat these biases in your own research, it’s helpful to space out the scheduling of interviews, ask your colleagues to join you during interviews to provide additional opinions, and take careful notes.

Combating bias as a UX designer

Although having biases is normal, it’s essential to try to eliminate bias from your research process to get the most accurate understanding of your users’ needs. Knowing the types of biases that exist and how you can avoid them will help you recognize when it’s happening, so you’re already off to a great start!

If you’d like to learn more about biases in UX research, check out this article on overcoming cognitive bias in user research from Design at NPR.

Sources: Google UX Design Professional Certificate

Coursera | Foundations of User Experience (UX) Design