Overcoming participant recruitment challenges in UX research
Strategies for low budgets and niche participant groups
Recruiting representative users is crucial, as it ensures that our findings reflect our target audience's behaviors, needs, and attitudes. So, one of the most frustrating pain points we encounter in user research is the inability to recruit users for our studies.
In particular, the two biggest roadblocks that get in the way are:
Low Budgets — Insufficient funding limits the ability to offer attractive incentives, reducing participant interest and participation.
Difficult to find users — Certain types of users, whether due to being from a rare population or their general unwillingness to participate in research, are difficult to find and recruit.
Depending on their circumstances, a research team may experience one, both, or neither of these issues. Each of these scenarios influences how research teams should approach recruitment and study design. In this article, we'll explore how to navigate the challenges associated with each of the four possible scenarios.
Low budgets & Difficulty finding participants
By far, the two most common pain points we hear about recruiting participants for UX research are low budgets and difficult-to-find users. Let’s go over what each of those means.
When discussing participants being difficult to find, it’s helpful to frame the conversation around incidence rates (IR); the proportion of people in the broader population who meet our requirements to participate. In the table below, you can see that if our target audience was for a mass-market application that anyone uses or even a broad segment of the population (e.g., people who use online banking), that would fall into the High to Medium tiers of IR.
On the other hand, if our target participants were from a more narrow demographic or interest group (e.g., optometrists, people who are currently pregnant, 3D developers, consumers of a niche product, etc.), that would fall into the low or very low IR ranges.
We find that most teams’ research tends to predominantly fall on one side of the spectrum; we’ll consider teams who do a majority of their studies with Medium-High IRs as primarily looking for ‘easy to find’ participants, while those focused on Low-Very Low-IRs as primarily seeking ‘difficult to find’ participants.
Defining what constitutes a low or high budget is less straightforward than measuring recruiting difficulty by IR. It is also somewhat situation-dependent; what is a generous budget for one team might not fulfill the basic needs of another. Generally speaking, if your team doesn’t have enough budget to source and compensate participants across all the research studies it plans or needs to do, we would consider that a “low or no budget” scenario.
When considering the various IR and budget scenarios, we get a 2x2 matrix that looks like this:
Regardless of which scenario you find yourself in, good research is possible — it’s a matter of making the most of your resources and leveraging techniques to minimize these difficulties and their consequences.
Scenario I - High IRs and Low Budget
Starting with this first scenario: a research team primarily recruits participants who are easy to find (high-IR), but they find themselves limited by low (or no) budget for recruitment. Budget constraints can be frustrating and restrictive, but groups in this situation can find success by using flexible and cost-effective solutions to engage their high-IR users:
Reduce costs with DIY recruitment - Using a recruitment agency or panel company can save a lot of time, but you pay for the convenience through commission fees. Techniques and channels like guerilla recruitment, social media, forums, going to a busy location your users frequently visit, and implementing in-product nudges are more time-consuming but often save significantly on costs.
Turn one participant into many - Reduce the need to recruit from scratch for every project by establishing a participant list. Do so by including an option to opt-in to future research at the end of every study you run; over time this reduces the costs and time needed to find willing participants. Additionally, consider the snowball technique by asking participants if they know another person who would be interested in participating.
Right-size monetary incentives - IR and participant honorariums have an inverse relationship. Fortunately for teams in this scenario, high-IR participants generally come with the most modest honorariums. Still, you’ll want to make sure you’re paying participants a fair rate. Consider modifying your research methods (e.g., use more unmoderated methods, shorter study durations, etc.) for cost efficiency.
Consider alternate incentives - For teams with particularly low budgets, it might be wise to consider a raffle-style drawing or non-monetary incentives like swag. If you opt for these incentive methods, relentlessly reduce the time it takes for a participant to complete your study — it’s a lot easier to get someone to answer 2-3 questions than join an hour-long call if you’re only offering a free shirt.
These techniques help teams maximize their budgets by reducing the costs associated with locating participants at the expense of a researcher’s time. Over time, you may be able to move from Scenario I to Scenario II by showing value through discount methods and articulating the potential impact of an expanded budget.
Scenario II - High IRs and High Budget
In this scenario, neither pain point is present; target users are easy to find (high-IR), and there is ample recruitment budget. This is the most manageable of the four situations and allows for the greatest research flexibility. Teams in this situation should maximize the returns on their budgets and consider:
Third-party panels - Unlike Scenario I, teams in Scenario II can make the cost-time tradeoff associated with employing panels. With high-IR participants, most reputable panel companies and tools with built-in recruitment functionality are viable options.
Use funds wisely & Negotiate - When using third-party panels and recruitment vendors, take advantage of bulk discounts or more favorable terms in ongoing service agreements. Small negotiations can yield large differences in what you get for your money here.
Set appropriate incentives - Test different incentive levels to find what your target audience finds most appealing. Often, high-IR users engage well with modest compensation, so be fair, but don’t overspend.
Without the pain points of budget and sourcing difficulty, most research methods are feasible; even very high-sample size rolling research programs are doable given how easy it is to find and compensate participants in large numbers.
One last caution: carefully monitor your data quality — use best practices in your study screeners and exercise due diligence in reviewing the participants sourced by third parties.
Scenario III - Low IRs and Low Budget
In Scenario III, we have a team with both of the most common pain points; their users are rare (low-IR), and they receive little budget to allocate to recruitment. Again, good research is possible here, but teams have to use a variety of strategies, invest a good deal of time, and sometimes make a few concessions:
Invest in building a panel - Third-party panels are most likely out of the question; lower-IRs have significantly higher costs per participant (in terms of both the incentive paid to users and the commission the sourcing agency charges). Many teams in this scenario turn to building and managing their own panel of users who have agreed to participate in research. This solution is an up-front investment that helps reduce the ongoing costs of doing research.
Tap existing contacts - A wise way to fill an internal panel (at almost no cost) is to coordinate with other teams who regularly interact with users in their day-to-day roles, like sales or CS in SaaS businesses. Use these natural touchpoints as a mechanism to invite users to your panel.
Individualized recruitment - Given the scarcity of target users, you may need to contact potential participants individually. That can involve searching for your target criteria and sending individual messages on platforms like LinkedIn. This is a time-consuming approach, so be sure to pair it with opt-ins for future research and snowball sampling.
Appeal with unique benefits - With limited funds, you may need to attract these participants with non-monetary incentives. Many users may be motivated by the opportunity to help shape the direction of products they use frequently for work. Others may find discount codes or free subscriptions a great alternative to a cash incentive.
Make participation easy - In addition to setting the right form of incentive, you will want to make it as effortless as possible for people to participate. That might include using unmoderated techniques more often, shortening study durations by relentlessly prioritizing questions and tasks included in the study, and even setting any moderated sessions outside normal working hours.
While the combination of these two pain points presents significant challenges, teams can still conduct meaningful user research that drives product strategy — but there might be a few methodological concessions. For example, high sample sizes are unlikely in this scenario, so you may primarily conduct qualitative methods. Using secondary research and inspection methods at the start of any project can be a helpful supplement that will prepare you to make the most of available test sessions.
Scenario IV - Low IRs and High Budget
In this final scenario, teams face the challenge of recruiting difficult-to-find (low-IR) users, but they receive the financial resources (high budget) to recruit participants effectively. A few things these teams should keep in mind:
Specialized panels - Though expensive, panels that have the established ability to reach your niche target are a time-saving solution that can be very attractive when you have the budget to employ them.
Cost-benefit analysis for panel building - Assess the long-term financial implications of using a specialized participant recruitment vendor vs. the investment required to build an internal panel. Some key variables you’ll want to consider are participant incentive costs, sourcing fees, ongoing maintenance expenses, panel attrition, and your overall research throughput. You might find it makes sense to use both, engaging a niche recruitment vendor and supplementing with your own participants.
Competitive incentives - To attract the right users to your studies higher-than-average incentives will be necessary. However, this comes with two implications:
Watch for bad actors - High incentives are attractive to fraudulent participants. Watch for bad actors who game your screeners and consider alternative/supplementary methods of verifying qualification.
Must-show ROI - This type of research is expensive, so there is extra pressure to show the ongoing return on investment. Without showing the value of research, teams can move from Scenario IV to III very quickly.
While a generous budget can facilitate niche recruitment, teams should still allocate funds carefully and demonstrate the impact of their research. An additional implication for teams in Scenario IV is that quantitative methods are still feasible, but the likelihood of achieving extremely high sample sizes on an ongoing basis is reduced.
The bottom line
Recruiting representative users for studies is a core research operations consideration for UX and product teams. Two barriers commonly impede this task: 1) Low budgets & 2) Difficult to find (low-IR) participants.
Depending on their circumstances, teams can have one, both, or neither of these pain points. This results in four unique scenarios, each with its own set of strategies and viable recruitment solutions. Using the framework provided in this article, you can determine which scenario best describes your team and use the suggested solutions to address your specific recruitment challenges.
Drill Deeper
Depth is produced by Drill Bit Labs, a consulting firm on a mission to advance the field of user research. We work side-by-side with UX and product design leaders to elevate their digital strategy, delight their users, and outperform business goals.
Our consulting services help make our free industry reports and articles possible. We’d be happy to open a conversation to discuss your specific needs and goals for this year.
Recent Podcast: Making Sense of UX Research Job Leveling
Lawton Pybus and I joined Amy Santee on the What is Wrong with Hiring podcast to drill into our research on UXR leveling frameworks, from IC to manager roles. Listen to the episode for a live discussion exploring the implications of our recent Data-Driven Career Ladder series.