Whether it’s your website, email marketing templates, or landing pages, user testing is a critical step toward the end of a digital redesign process. Implemented properly, the process should include live and remote user testing utilizing one of several tools made expressly for those purposes. But the reality is that process requires time and adds costs that many arts organizations simply can’t accommodate in their budget.
Fortunately, we’re here to help and have put together a remote user test that’s capable of providing user feedback that will help you maximize conversions. This article is going to provide you with everything you need from concept to the actual form questions, to make that happen.
Remote User Testing Defined
Remote user testing is just a fancy name for gathering feedback via a form or similar tool. This process is unmoderated, which means you tend to get a higher degree of (brutal) honesty. But for remote testing to produce useful results, you need to gather more info than user feedback on content and user interface. You’ll need to know their device and browser along with attaching clearly defined conversion goals to each question.
Parts Of The User Test Form
Tech Info
This is a crucial, but often overlooked, component of user testing. Without this information, it won’t be possible to reproduce that user’s experience in order to replicate errors or experience their journey firsthand.
- Device Type
- Device Model
- Operating System (OS)
- Browser
Pro Tip: you don’t need the actual devices/browsers to reproduce the user’s experience. Most modern browsers provide reliable device emulators as part of their developer tools, such as Chrome’s device emulator and guess who has an article that walks you though how to use it?
User Info
- Name
- Email address
You can include additional fields but those are the bare minimum which allow you to reach out to the user for clarification if needed. So, while you can add additional fields, keep in mind that less is more when it comes to getting as many users to submit the form.
Questions
Once you know who the user is and how they’re experiencing the site, it’s time to ask them questions. There are two primary types of questions to ask and you can use one, the other, or a combination of both.
Close-Ended Questions
These are straightforward yes/no or multiple-choice style questions.
- Pros: easier for users to complete so you’re more likely to get a higher number of responses.
- Cons: limits your ability to determine if users are processing keywords via content architecture and copy writing.
Examples:
- Q: Is there an event on October 18, 2020?
A: Yes | No - Q: Your company is interested in becoming a corporate donor and you need to see which giving levels are available. Which of the following is NOT an existing giving level?
A: $2,500 |$10,000 | $25,000 | $75,000
Open-Ended Questions
These questions require more than simple or one-word style replies. Multiple select lists along with single and paragraph text field questions are the most typical options.
- Pros: easier for users to complete so you’re more likely to get a higher number of responses.
- Cons: limits your ability to determine if users are processing keywords via content architecture and copy writing.
Examples:
- Q: What is the name of [ORGANIZATION NAME] Director of Development?
A: single text field response. - Q: You lost a ticket to an upcoming event, what does [ORGANIZATION NAME] recommend you do?
A: single text field response.
Designing Questions To Accurately Measure User Scenarios And Conversions
First and foremost, every question in your remote user test form should be designed to measure the effectiveness of your key conversion goals.
- For most performing arts organizations, this will include ticket sales or memberships. For groups that don’t focus on earned income, online donations will be the primary revenue goal.
- Secondary goals include social shares and form submissions (i.e. lead-gen).
In addition to conversion goals, you want to measure how well users are finding and internalizing key information, such as guest artist names, run dates, starting times, run times, directions, contact info, etc. Open-ended questions are an ideal way to measure how effective your content and page layouts are at delivering that information.
For example, which of the following two versions of the same information does a better job at providing the key run date and run time info?
Writing Effective Questions
Ideally, you want to write questions in a way that engages the tester in a specific user story or scenario. Instead of “go here, do that” consider an approach that encourages the tester to see things via a user’s motivation.
Here are some examples that describe the scenario as a story instead of overly direct, leading-question style phrasing.
- Good: You want to bring a friend to an event between [DATE RANGE] but both of you are on a budget. What is the least expensive ticket price you can find for all events within that time frame?
- Bad: Visit [URL] and confirm the lowest ticket price.
Pro Tip: effective questions will include a mix of those that provide a direct URL to the page in question and those that require the tester to use the navigation menu.
Remote User Testing Considerations
- Cross domain testing. This is more common than not and exists when an organization moves site visitors to a different website to complete a conversion process, such as purchasing a ticket. In these cases, you may or may not want to configure questions that force users across that threshold, or you may want to create mutually exclusive tests per platform.
- Don’t forget to offer testers something besides heart-felt thanks. Simply put, their time isn’t cheap so you shouldn’t be either.
- Include a free form paragraph text field question that encourages the tester to offer up any additional insights or observations.
- Include a clear deadline (date and time) when testers need to submit replies. Keep the test window as short as possible and if timing permits, include week and weekend dates.
In the end, it’s important to remember this article encompasses an approach to user testing that provides meaningful, but minimal, actionable results. If you have the time and budget to engage the more involved processes, absolutely take advantage of it.
Resources
Because ArtsHacker is all about providing you with tools, we have an example user testing form that includes both open and closed ended question types. It also includes a list of responses from the tech info questions, such as specific device types and browsers. Since tech never stands still, you’ll want to remember to update that list over time with the latest options.
In addition to the Google sheet version where you can copy/paste content into your form tool of choice, we have a special treat for WordPress users who also use Gravity Forms. You can download the entire sample form, settings and all, via the following Gravity Forms json file (import instructions). It only uses default form fields but you could get fancy by replacing most of those with survey extension powered fields and bask in the warmth of auto-tabulate responses.
Form Questions In Gravity Forms json Format
Form Questions In Google Sheet Format
While this form is geared toward website user testing, you should be able to modify the questions in order to use on landing page or email marketing template redesigns.
