competitive
testing
Time
⏲
7+ Days
Difficulty
đź•ą
Hard
Materials
📦
Recording equipment
Spreadsheet to track responses
People
đź•´
2 Reserachers
5+ Users
Overview
Competitive user experience research focuses on how people use and perceive a product or service. Unlike traditional competitive analysis, it does not address the relative popularity, business model, or revenue stream. Business competitive analysis or brand analysis tries to understand what makes companies successful by looking at their revenue and other “fundamentals.” Aside from the occasional feature grid or brand perception study, such analyses rarely address the consumer’s or user’s view. User experience research can work in concert with more traditional competitive analysis, of course, but its focus is from the bottom up rather than from the top down.
What
While it can feel good and raise morale to identify problems in competitor’s products, the real goal of competitive user experience research is to figure out how to creatively differentiate your product from the competition—not just fix other people’s mistakes.
Michael Hawley quote: “Enough understanding of competitors’ offerings to recognize opportunities to set my designs apart, yet minimize the influence of competitors’ designs on my thinking.”
Competitive research ignores the assumptions or constraints under which the other products were constructed and concentrates purely on the user’s perspective.
When
Competitive research can be done at any point in your product development cycle. Picking the perfect moment to do competitive research is less important than doing it regularly and thoroughly. For the most benefit, you should check on your competition repeatedly throughout the lifetime of your product. Whenever you’re not testing your own work, you should be researching your competitors’.
Competitive research is most significant within the development process:
When producing requirements
Research into what the competition’s customers find useful, what they find attractive, and where those products fail can guide selection and prioritization of features for your own product.
Before you redesign
As your product develops and evolves, competitive research can answer questions about design directions. Competitive products can be treated as functional prototypes of some of your own ideas.
When your competitors make major changes
Researching the reasons for those changes and how they affect people’s perceptions and behavior helps you decide how to react. Analyzing the results of those changes can keep you from blindly imitating decisions that may not be improvements.
Steps
Step 1 Identify and profile the competition
This may seem obvious, but you need to know who your competitors are before you can start to analyze them. Part of the work of identifying the competition is getting over your own beliefs about what other people will think your product does. The easiest way to start looking for potential competitors is by searching the internet. Make a list of three things your company or product does. Try searching all those phrases. You’ll probably find that different searches return different companies. Make a note of all the companies and how you discovered them. Keeping track of what search terms brings up potential competitors will help you keep track of how your competitors promote themselves. Once you have a list, prioritize it.
Tier 1 competitors: These are your most direct rivals. They will be the primary targets of your analysis. Your list should contain no more than 5 of these. Even if you believe your product is completely new and innovative, its’ still battling something for your audience’s attention. That “something” is the competition.
Tier 2 competitors: Products from the same category as the Tier 1 competitors. Either they are not as directly competitive or they are largely identical to a Tier 1 exemplar. Remember, you’re not aiming for ultimate completeness, but rather a useful sample of competition that will help you map out the landscape within which you’ll make design directions. Your list shouldn’t be more than five to ten of these.
Niche competitors: These compete directly with part of your product, but not the whole thing.
Step 2 Profile the competition
A complete competitive analysis doesn’t just name competitors; it includes competitive profiles of all the Tier 1 competitors. These consist of two elements: a description of the product and a profile of its audience.
Step 3 Product description
A product description should not just be a list of features. It should be a statement from the users’ perspective of the value it brings them. Product descriptions should not be more than a few sentences long.
Step 4 Audience profile
What kinds of people use your competitors’ products? You probably won’t be able to get detailed statistics, but with a little research, you can make some good guesses. The way you categorize the audience will depend on the product or service. What categories seem most relevant to outside analysts? How do people seem to identify themselves on forums and blogs? Then, you write the profile. The audience profile only needs to be a few words or sentences.
Highlight the differences between these people and your audiences. If they’re direct competitors, the profiles could be quite similar to yours, but each of them will have at least one key element: the reason why they’re using someone else’s product. Why? What makes the competition’s user market different from yours? You may need to create separate profiles when you feel that the product serves multiple audiences.
Step 5 Define a set of key dimensions for comparison
The most important step in analyzing your competition is creating a set of dimensions—a framework within which you can compare your competitors. In defining a set of dimensions, you are more importantly liming the scope of the competitive analysis. You are limiting it first to what your users may consider important, and you are limiting it secondly to those dimensions relevant to the research questions you need to answer. Dimensions are categories drawn from specific types of attributes. They can come from specific features or more abstract qualities.
Step 6 Dimension identification
This should come from the user’s perspective. There are two ways to define dimensions—by asking users and by looking for yourself. You should do a combination of both.
Step 7 Feature audit
Sit down and make a list of prominent features of your product. From a user’s perspective, what does your product do? What are the prominent functions? What are its attributes? Then, look at each of your competitors’ products and check which ones have the same or similar features (measure similarity from the perspective of the user).
Step 8 Prioritize the features
Combine the dimension identification and feature audit lists and prioritize the features. Prioritize based on the areas that mean the most to the product’s functionality, to the company’s success, or to the user’s satisfaction.
Step 9 Collect important attributes of competitive products
Attributes aren’t necessarily related to functionality; they make a product memorable, likable, or interesting. Attributes describe the feeling and environment of the user experience.
Techniques
Competitive analysis
The same techniques that provide insight into your product—usability tests, interviews, focus groups, surveys, etc.—work as well when applied to your customers. The major difference in approach is the amount of focus you place on creating a balanced perspective. In competitive research it’s as important to understand what your competitors have done right as it is to know where they’ve faltered.
>1 week: Concentrate on getting the most out of existing sources: Industry analysis, whitepapers, newspapers, blogs, user forums, etc.
Recruiting
Keeping the identity of your company anonymous is important when competitively recruiting. Who you recruit depends on what type of research you want to do (e.g., usability testing advanced features, focus groups of current users, or research with inexperienced users).
Competitive product interviews and observation
One-on-one interviews with users of a competitive product can reveal much about what makes that product functional and where it fails. Watching people use your competition’s product reveals usage patterns that your product can emulate or avoid.
Focus groups
Inviting the users of a competitive product to participate in a focus group can reveal why they use it and which aspects of it attract and repel them. Such groups can also reveal people’s views of the brand and the identity of the product, and the qualities they assign those brands. More than one product can be discussed in a competitive focus group. These groups can be made up of people who have experience with one of the products or with people who have no experience with any of them. When presenting multiple products, the order in which they’re discussed should be varied so as to minimize the bias that can come from having the same product shown first every time or having the same two products appear in the same order every time.
Surveys
Create reasonable objectives about what your survey is to accomplish, focus most of the questions on the higher priority issues, and present the features in a neutral way. Popular topics of competitive investigation in surveys include the following: other products your users use, how much they use the products, their loyalty to the competition, and the features they use.
Analysis
Unlike the goal of a traditional competitive analysis, which is to create a side-by-side, feature-for-feature comparison grid, point by point matching of features isn’t always important for competitiveness from the user’s perspective. There is usually a core set of functionality that needs to be similar for two products to be considered competitive, but beyond that the actual feature set can vary broadly and the two products can still compete. Competitive user experience research should reveal the fundamental strengths and weaknesses of the competition, not create a scoreboard.
Once you have the reports from the data collection phase, take the report and enumerate the advantages that the competition’s product gives their users and the hindrances it places in their way. Consider both the collection of a company’s strengths (portfolio of core competencies) and the weak points in the competition’s approach in creating experiences (portfolio of frequent foibles).
Action
Once you’ve collected, analyzed, and compared the data, you need to make the research usable, to create actionable intelligence. To do this, refocus on your project. Look at your list of advantages and hindrances and reflect them back on your goals. For every item on the list, explain its relationship to your product or service.
Every advantage a competitor’s product provides can be seen as a threat to yours, and every problem an opportunity. Then, create a theory explaining the motivation behind the design choices that were made by your competition. Why was the product made the way it was? What were the constraints? What was the likely rationale? What choices do you agree with? Which do you not? Which are inexplicable? Concentrate on the whys of your competitor’s decision-making process, not necessary on what was done. The final outcome of competitive research is a deeper understanding of what makes a good user experience, admiration for your competitor’s ability to solve problems, pride in your own product, and a plan that allows you to use this knowledge to your advantage.
Resources
None
Tools
None