Listening to users considered harmful?

Is "listening to users" really the most important way to keep them happy and -- if we're lucky -- passionate? Is giving users what they ask for the best way to help them kick ass? Or should you create or modify a product based solely on what you believe in... even if it doesn't match what users tell you?

Last weekend I attended the sold-out Parelli Natural Horsemanship conference. I was surrounded by 2000 passionate fans (at least 75% of the people were wearing at least one Parelli-branded shirt, jacket, or hat). The conference was amazing (more on that in another post), but the real reason I went was to interview the founder/visionary Pat Parelli, for the Creating Passionate Users book. His hugely successful, multi-million dollar company is one of the few we've found that does virtually everything on our "reverse-engineering passion" checklists, without having first waited for the fans to do it themselves.

In the equestrian world (total annual impact of the horse industry on the US economy is $112 billion [yes, that's with a "b"]), Pat Parelli has so greatly outstripped the "horsemanship" competition that it doesn't even make sense to talk about competition. Software engineers will appreciate that horse training doesn't scale. So Parelli decided to teach others to do what he does, and of course sell those folks a ton of high-end equipment and training products to help them do it. Nobody -- absolutely no other individual "horse whisperer" or company -- comes anywhere near Parelli in size and scope of the business - Parelli has two training centers, one in Colorado and one in Florida (combined over 700 acres for the facilities), and hundreds of thousands of participants in the home-study programs, clinics, and club membership. Their Parelli Horseman's University is one of the only state-accredited "natural horsemanship" programs in the US.

So that's the backstory. I have weeks' worth of posts to make on what I learned from Pat about the ways in which they've become such a passionate user success story, but today's post is about something I had completely wrong when I interviewed him:

Me: "So, you've recently made drastic changes to your program--a program that was already extremely successful. It's obvious that you've been really listening to your members and taking their feedback and using that to make these sweeping changes."

Pat: "No, listening to our members was maybe 20% of it, but the other 80% was something else."

And then he said it:

"We changed our entire program because WE knew we could do better. Because WE were still frustrated that people weren't learning quickly enough or progressing through the higher levels as well as we thought they could. People still weren't having the kind of relationship with their horse that we knew they could have, even though our students were delighted with the progress they were making. So we changed it all."

It turned out that most of the major changes they made to their program came not from user requests and suggestions, but from the Parelli team's own innovations. He went on to explain that their members/students/users had no idea what was needed to make better, faster, deeper breakthroughs. In fact, many of the changes went against what their user feedback seemed to suggest. In other words, in many ways the Parelli team deliberately did not listen to users.

They trusted themselves, and did what they believed was right for their users, even if it meant doing things that on the surface seemed even less user-friendly.

Most of us realize that focus groups are notoriously ineffective for many things, but we still assume that listening to real feedback from real users is the best way to drive new products and services, as well as improve on what we have. But there's a huge problem with that -- people don't necessarily know how to ask for something they've never conceived of! Most people make suggestions based entirely around incremental improvements, looking at what exists and thinking about how it could be better. But that's quite different from having a vision for something profoundly new.

True innovation will rarely come from what users say directly.

This doesn't mean that you don't listen to users--because the truth is embedded in what they say...but you have to look for the deeper meaning behind what they ask for, rather than always taking them at their word. If they ask for "D", as an improvement to "C", you might have to dig deeper to find out what it is about "D" that they want. And in that answer, you might find the nugget that leads you--and only you--to come up with "S" as a solution. And the "S" solution looks nothing at all like "D", but gets to the heart of what users really wanted and needed when they asked for "D".

In the end, you might have to trust yourself, even in the face of users who either want more than you know would be good or something less or different than you know you can offer if you keep innovating in revolutionary--not just incremental--ways. Our Head First books are among the top-selling computer books today, virtually all of them occupy the #1 slot for their topic category. But not only did nobody ask for such a bizarre format for a technical book, we were warned that it would never work. We were told that people would hate these books. That they were too different, too pictorial, too... tacky to be taken seriously. But we knew the brain science and learning theory behind the format, and trusted that the principles worked. That for most (not all) readers, this format really did lead to faster, deeper learning. We trusted that people would look beyond the surface aspects of the implementation, and that if they got real results from the book, they'd tell others.

Two other publishers turned us down for the series before O'Reilly took the chance. And I was nearly fired from Sun for trying to sneak 5% of what's in Head First into Sun courseware.

Are users/readers too clueless to know what to ask for? Of course not. But it's not a potential Java programmer's job to be a learning theory expert, anymore than I could have helped conceive of the iPod. I could make incremental suggestions about most of the tools I use, sure, but I don't have the background, skills, or vision to suggest the kind of revolutionary changes that create breakthrough products and services outside of my own very narrow domain.

What sparked this post was a somewhat contentious (and bold) 37Signals post, but I also remembered this post by Wiley editor Joe Wilcox.

This is tricky, of course, because it's not always obvious which user complaints/suggestions are based on real problems with your product, vs. naive feature requests that would do more harm than good. (Don't forget the Happy User Curve)

And this is NOT about giving them simply what we know is good for them but that they really don't want, because they probably won't stick around. This is about giving them what they really DO want... but simply don't realize it because they had no way to imagine it.

So maybe the key is to listen not only to what users say, but more importantly to what is motivating what they say. The rest is up to us. If we really care about our users, they'll just have to trust us... but more crucially--we have to trust ourselves.