How to use UX surveys to gain product experience insights

What is a UX survey?

A user experience (UX) survey collects quantitative and qualitative data about a user’s interactions and experience with a website or digital product. UX survey data supports and complements website analytics and UX metrics collected through methods like A/B testing, heatmaps, usability testing, and session recordings

5 UX survey best practices

If you want to know how users really feel about their interactions and experience with your product so you can address specific pain points and prioritize product changes, a UX survey can give you the useful, actionable user data you need.

But here are five things to keep in mind before you launch your survey:

1. Avoid bias and leading questions

Before you launch your survey, check that you're avoiding bias. Here are a few things to look out for as you state your survey objectives and write your questions:

• Confirmation bias: seeking only the data that confirms your beliefs. For example, "Would you be more likely to click this button if it were bigger and brighter?"

• Framing effect (aka asking leading questions): asking questions in a way that prompts a specific answer from your users. For example, "What did you love the most about your experience in our product?"

• Unbalanced scales: similar to the framing effect, unbalanced scales will sway answers by limiting users' choices. For example, "How much did you enjoy your experience on a scale of 1 (enjoyed it a little) to 5 (enjoyed it a lot)?"

2. Ask the right questions

To get the data and user insights you need, focus your UX survey questions on the problem you’re trying to solve. If you don't know what the problem is yet, ask questions to help you identify your users' pain points so you can discover the blockers they're experiencing (and then remove them).

It's helpful to ask closed-ended questions early on, then—if you're using a survey tool like Hotjar—use Survey Logic to follow up with open-ended questions that explore the subject on a deeper level.

For example, let's say you want to find out how users feel about a new tool or feature. Some closed-ended questions you could ask are:

• Have you used [new tool/feature]?

• How easy was it to use [new tool/feature] on a scale of 1 (very difficult) to 5 (very easy)?

And you could follow up with open-ended questions like:

• What are your first impressions of [new tool/feature]?

• What is one thing you would change about [new tool/feature]?

3. Make it quick, keep it relevant

Make on-site surveys—surveys that pop up or slide onto the page—as relevant, quick, and easy-to-complete as possible. Start by asking one or two closed-ended questions to get quantitative data or to segment by user cohorts. If you follow up with open-ended questions, keep them simple.

Note: sometimes you need more detailed product experience insights, in which case you can invite users to participate in an external link survey. External surveys give users more time and space to thoughtfully answer more specific questions about the user experience.

4. Sample continuously and share your findings

Your team is never really finished building and improving your product, so you should never stop asking for feedback. But keep in mind that survey results can be impacted by external factors, like a disproportionate number of new or inexperienced users or failure to distribute representative cohorts to both versions of an A/B test. By sampling over a long period, you can reduce the risk of your UX survey being skewed by unique events.

5. Ensure survey cool-off periods

On-site surveys can distract users from their usual experience with your website or product, and multiple invitations to external surveys can become irritating. Plan cool-off periods between surveys to avoid interfering with the user experience and repeatedly re-surveying individual users.

How to conduct UX surveys: the Lean approach

It might be tempting to address every topic or pain point in one survey, but a targeted approach will give you relevant, actionable user insights at each stage of product development, helping you prioritize UX changes and improvements.

As we have with other pages in this UX guide, we'll use the Lean UX model to… guide us:

UX survey questions for the THINK phase

The THINK phase is about generating ideas and uncovering qualitative insights, which is a great time to ask open-ended questions:

Question: If you could improve one thing about this product, what would it be?

Why it’s useful: Get ideas for improvements and uncover possible user pain points.

Question: What is one thing you wish this product could do that it doesn’t do already?

Why it’s useful: Get ideas for new tools or features to add to your product roadmap.

Question: If you knew that we would make one change to our product the next time you logged in, what would you want it to be?

Why it’s useful: Get ideas for improvements, uncover possible user pain points, and prioritize changes.

UX survey questions for the MAKE phase

Using UX surveys in the MAKE phase can help you prioritize aspects of your product roadmap:

Question: On a scale of 1-10, how would your use of our product be impacted by [feature/change]?

Why it’s useful: Understand the potential impact of changes from the users' perspective.

Question: Please state your agreement with the following: "[feature/change] would make my job easier."

Why it’s useful: Understand how users perceive potential additions or changes to your product in terms of their goals and objectives.

UX survey questions for the CHECK phase

The CHECK phase helps you understand whether product changes have improved the user experience. You can use a mixture of closed- and open-ended questions during this phase:

Question: Rate your agreement with the following: "[feature/change] has made my job easier."

Why it’s useful: Assess the performance of a new feature or product update in terms of users' goals and objectives.

Question: How has [feature/change] affected the way you use our product?

Why it’s useful: Get voice of the customer (VoC) feedback that directly relates to a recent change or update to your product.

Question: Is there anything you would change about how [feature/change] works?

Why it’s useful: Gain product experience insights about new features and changes that can lead into the next THINK phase of the Lean UX cycle.