top of page

Research, Evidence, and Guts


One of the pillars of agility is feedback from customers and prospective customers. When I worked for JSTOR, we were (they are) blessed with millions of loyal customers, one of whom had their logo tattooed on his arm. Loyal customers might be willing to take time from their busy days to give you feedback -- especially if you pay them.


Unfortunately, not every company has such a large customer base or the deep pockets to pay people for their time. They often rely on a few highly loyal customers to provide feedback. Even companies with larger user bases like JSTOR need to worry about echo chambers -- doubly so for small organizations.


Some evidence is better than none. Doing the research is an organizational imperative. Not spending money on research is pennywise and pound-foolish. The alternative is to go with your gut and build products, hoping you got it right. The cost of building that product is an important consideration when weighing the cost of the research. One thousand dollars worth of Amazon gift cards is the same as the cost of one developer for two to three days. It is an easy decision for me!


You can learn a lot from ten hand-picked interview subjects. The more you pay, the more likely you are to get a diverse set of subjects with different backgrounds and levels of affection for your product. As a bonus, if you use social media to recruit subjects, you also get the increased exposure that money always brings.


There are other ways to gather evidence. If your product is software-based, you can collect usage data. When you build new capabilities into a software product, part of that effort must include evaluating what feedback will be helpful. If users are making choices about how they use your product, you need to be capturing those decisions.


No matter how much evidence you gather, at some point, you must form hypotheses about what it means for the product. Great teams enjoy diverse viewpoints from team members. Creativity stems from exploring different options and combining them in various ways.


Research techniques provide well-defined methodologies for setting up experiments to isolate and test various hypotheses. Being mindful of expectations of user behavior will inform where you put test points in your system. Even modern hardware systems collect and use diagnostic data to evaluate usage and performance.


Using evidence to design and test products is part science and part art. Great teams understand both and use the full power of all the minds on the team to explore as many options as possible. You miss a tremendous opportunity if only one or two team members make all these decisions.


Recent Posts

See All

All Great Leaders Have Empathy

It’s ironic that when we’re interviewing for leadership roles, we tend to prefer people with a strong presence and confidence. Yet, the greatest leaders in modern history -- people like Gandhi and Man

bottom of page