Interpreting diverse feedback
How to find patterns and reconcile conflicting feedback.
In the last 3 months, I’ve been teaching at Cornell Tech in a course called Product Studio. While reflecting, I observed a challenge that many product managers face, interpreting feedback.
As product managers, we seek and receive feedback from a wide variety of people: customers, potential users, coworkers, managers, investors. The list can be very long. But research has shown that “feedback from diverse audiences may contain contradictions, focus on different topics, and vary widely in structure making it hard to find emerging patterns, reconcile conflicting ideas, and prioritize revisions.”
How do you find patterns and reconcile conflicting feedback?
In interviews with more experienced product managers, three consistent themes emerge on how to deal with feedback.
Evaluate the person giving feedback
You’re more likely to accept and agree with feedback from a person you respect. Respect is an evaluation judgment of the person giving feedback. Experienced product managers evaluate in a structured manner, the qualifications and quality of the feedback. You can ask questions that help you weigh the feedback given such as:
* What are the qualifications or experiences that increase or decrease the value of this person’s feedback? [Identify a list of pros and cons]
* What is the level of detail (i.e., quality) in the feedback given? [Identified specific issues versus general statements of like or dislike]
Are the techniques you used to evaluate the person giving feedback? Share how you determine if the feedback is valid when talking to “experts.”
For those that are more mathematically inclined, you’re developing a weight value (0-1). This should be applied to discount the feedback received.
Categorize or group feedback from multiple people
When we hear conflicting feedback from multiple people, the natural inclination is to collect more feedback. Collecting additional feedback is an opportunity to get “the right answer” (e.g., 8 out of 10 people giving feedback all said XYZ). You can categorize and group feedback with the same sentiment to reach this result. This works in situations where there is a single “optimal” answer, a bit of wisdom from the crowds.
Yet, this technique only works if there is a single “optimal” answer. This is not true in all situations, where there may be multiple valid options. For example, in cases of personal preferences (e.g., favorite color), there is no single “right” answer. In those cases, seeking additional feedback doesn’t resolve the conflicting feedback. It actually will waste time. Instead, you must recognize that each grouping of feedback is valid because you have multiple “right” answers.
Deciding and identifying next steps
Evaluating and understanding feedback is important, but only as important as deciding and identifying what you will do with the feedback.
Many product managers over-index on feedback collection and analysis. What they miss is deciding what they would do with the feedback, getting stuck in analysis-paralysis. In most cases, a lack of time suddenly forces decisions.
More experienced product managers take the opposite approach. They decide a default action, prior to receiving any feedback. Then, after collecting feedback, they decide whether the feedback is sufficient to change their default plan.
This approach solves two problems:
You always have the next steps. It is the default action so you aren’t paralyzed by feedback.
You interpret feedback as a way to amend your existing plan. The feedback is never in the abstract.
For more junior product managers, it can be difficult to create a default action plan in the face of no feedback or data. I plan to write more about this in the future, exploring how to develop the skill of deciding an action when you are uncertain and have no data.
Like or dislike this article? Leave a heart.
Sources: