« Be Provocative | Main | Why they don't upgrade (and what to do about it) »

Screaming users considered good

Screaming_womansmall1

We all know our users can have really strong opinions about our stuff.  And we all know we need to listen carefully.

My very favorite user comment of all time came from a young woman who had been using this cool design tool (something I’d built) for about 12 weeks.  At the end of the project, I was doing the standard debriefing of the users, asking what they liked and didn’t like, what worked, what didn’t work, what was frustrating… the usual sort of post mortem on a project. 

She told me that my software, my baby, the thing I’d been working on for the past 2 years was “..the most white male fascist tool I’ve ever had the misfortune to use…”

I was somewhat taken aback.

“Ah, yes…”  I stalled for time, desperately trying to hold it together.  “And what made you feel this way?”

The conversation went on for some time after that (as you can imagine).  And while it was a painful episode, it was a really valuable learning experience. 

Although I knew intellectually that not everyone would see my system as I did, I was floored by her reaction.  But it made the point: as a designer, you really have to be aware of other folks opinions (even when they don’t jibe with yours), and you have to know what it is you’re building.  Sometimes, your product is going to passionately piss people off.  Sometimes, that’s okay.  In many cases, you simply can’t design a product that will make everyone sing your praises and want to send you roses.  I love my iPod, but I know there are some people who think it’s devil spawn.  If Steve can’t get everyone to love his things, I’m not sure I can. 

So I’d succeeded in creating a passionate user.  Sadly, it was passion in the wrong direction. 

After I recovered my composure a few weeks later, I realized I was really glad she’d told me.  The ten users I’d interviewed before her were all pretty nice and even-keeled.  “Oh yeah… it worked well…”  or even the sweetly positive comment  “I could do things with it that I could never have done before.” 

But in retrospect, I didn’t learn much from the nice folks who told me everything was fine and ducky.  I did learn a great deal from the ones who struggled, my users that just didn’t get it, had really strong reactions or failures. 

As Henry Petroski writes in To Engineer Is Human: The Role of Failure in Successful Design, we learn more from our failures than our successes.  But only if we pay attention to the failures and figure out what to do right the next time. 

The trick is to figure out what the message is from the user.  I did have the presence of mind to ask her what “fascist” meant.  Sure, I know the dictionary definition, but I couldn’t figure out what it meant in the context of the tool I’d built. 

My question opened up the sluice gates and I heard an awful lot about “not letting the user have a choice” and how our design tool “forced the user to do things in a particular order.”

Gee.  We did it that way because we knew it was more efficient.  But provably correct didn’t win the heart and mind of this user—she did things in a different way, and the tool was forcing her to go along a different path.  It felt fascist to her. 

Okay.  Got it.  So it wasn’t the Gestapo of all software, but it really was at variance with her approach.  In an instant it became clear what we could do differently the next time around. 

Bottom line:  Every product evolves.  It’s the rare (or trivial) that gets it right the first time and sticks with it for the rest of time.  Listen to the screamers and whiners and people writing nasty blog posts.  It’s painful and tough, but worth it. The screamers may not know it, but they’re really helping you out with the next release. 

Posted by Dan Russell on September 18, 2006 | Permalink

TrackBack

TrackBack URL for this entry:
http://www.typepad.com/services/trackback/6a00d83451b44369e200d8342c0f2353ef

Listed below are links to weblogs that reference Screaming users considered good:

» Deal critically with criticism to protect motivation from Loud Thinking
Criticism is one of the most powerful creators of progress in every facet of life. People change, laws change, governments... [Read More]

Tracked on Sep 20, 2006 3:24:29 AM

» 认真的对待批评以保护前进的动力 from dhansson
David通过两个小故事,来说明对于程序员而言,面对外界批评的时候要勇于面对和接受,消极逃避只能扼杀创新. [Read More]

Tracked on Sep 20, 2006 9:10:33 AM

» QuickLinks round 3 from StupidApplicationException
Kathy sierra writes about user responses and learning from design failures - user quote: “..the most white male fascist tool I’ve ever had the misfortune to use…” [Read More]

Tracked on Sep 21, 2006 2:53:05 PM

» Passionate / screaming users from Kirill Grouchnikov's Blog
Kathy Sierra writes on her blog that the best feedback you can get is a negative feedback. I fully agree, but with one condition - if the feedback is concrete (it doesn't have to be constructive). You can download this... [Read More]

Tracked on Sep 23, 2006 12:58:31 AM

» Reclaiming Fascism: perspective please, people! from Stilgherrian
No, this isnt an apologia for Nazis, far from it. Its a plea to reserve fascist for situations which actually warrant the term. There may (may the gods forbid!) come a time when we need to label a government fascist and b... [Read More]

Tracked on Sep 23, 2006 6:34:45 AM

Comments

At least you took the time to try to understand your user's complaints, rather than ignore them or, worse yet, badmouth the user. My pet peeve? Asking for help with a problem, complaining when I get no response, and then still getting no response. I won't use PubSub for this reason. If they'd listened and taken my request for help with their website seriously, I'd have been great word-of mouth advertising. As it stands, I won't use their service and I'll turn away anyone else I know who might want to try it. Your complaining user may not have liked your product, but she'll definitely give you a fair shake in the future . . . all because you listened.

Posted by: Mary Warner | Sep 18, 2006 8:59:27 AM

This whole post also applies to change management (and communication in general). I find that when people are really upset, fighting a change, or generally difficult to work with - it's often because they have needs that have been overlooked, minimized, or ignored. It always helps to listen to the screamers. Sure there is a percentage that are screaming without adding any value (just for the "fun" of it)- but most of them have a point!

Posted by: ann michael | Sep 18, 2006 9:20:45 AM

I always remember the things that people yell at me. It really is effective. Your story was very instructive.

Posted by: Rhea | Sep 18, 2006 9:51:07 AM

My mephisto-based blog doesn't have trackback support yet, so I'll just have to give you the URL I'm afraid:

http://vagueware.com/2006/9/18/awful-users-mean-great-product

Great article, and it reminded me of the simple fact that compliments really don't get you anywhere further down the road other than "you're great" - if you need to reinvent, develop, challenge, move along, you need to hear the bad stuff and loudly.

Posted by: Paul Robinson | Sep 18, 2006 10:53:42 AM

Once I'm able to pick myself up off the floor when I receive a nastygram, it's not usually until several days later that I can really look to see if there's something I can learn from.

Praying for thicker skin,
Cyndi

Posted by: Cyndi L | Sep 18, 2006 11:07:15 AM

Ugh, you've caught me. I was sitting here smugly nodding and thinking of the users that I know I've converted from problem customers to big fans just by taking their yelling seriously and by talking them through what upsets them and what we can do about it. But mostly, that was before I was in development.

It seems like the more I develop, the more defensive I get and the more likely I am to respond to those people by letting the users know that it was user error and not programming error that caused their problems.

See me blush. I know very clearly that approach will never create a passionate user.

Posted by: Megan | Sep 18, 2006 11:10:45 AM

"All feedback is a gift no matter how poorly it is wrapped." - I heard this saying a long time ago and really have tried to live by it. Your article helped me dust it off and move it back into my active belief system.

Posted by: Joe | Sep 18, 2006 11:11:27 AM

Heh... this reminds me of when I created a YahooGroup composed entirely of users who were very angry and upset at the product I was the lead designer for. It shell shocked the developers who participated, but it was by far the most useful and valuable input I ever got... and it had the side effect of turning most of that group into advocates. -jk

Posted by: John K | Sep 18, 2006 11:11:59 AM

I remember the first time I had a software project that I thought was simply amazing and the users thought it was the biggest pile of horse manure they have ever seen. It was certainly tough to sit there and hear them tear my hard work and wonderful ideas to shreds, but it also taught me to pay more attention to what they want in the first place. At the time I was younger and knew that I knew better than them what they wanted. Now I realize I don't.

Posted by: CobolGuy | Sep 18, 2006 11:37:06 AM

What was it about your software that indicated to her that it was white, or male? Or was it just that YOU are both of those things?

Posted by: Michael Campbell | Sep 18, 2006 11:38:12 AM

One of the best nuggets of wisdom I have been given was that hate is not the opposite of love, apathy is. People who hate something are often angry because they want to like it, feel like they should like it, but something is getting in the way.

In Cathy-speak, perhaps we could call hate "direction-inverted passion", because often it involves people who care a lot, and are trying to make things better. Of course, there are always people who are just whiners in general, but they are few and far between.

If you can see understand that apathy is far more harmful than passion. Don't take it personally, because usually they don't hate you, they hate the problems. It is just a little threatening to hear the truth.

Morgan

Posted by: Morgan Goeller | Sep 18, 2006 11:58:48 AM

Yeah that whole 'the customer is always right' thing goes out the window when you're faced with criticism.
Our biggest flaw is having so many cool ideas for things, we sometimes get a little over-excited and forget that the customer only wants some simple little feature. It's tough to get bad feedback, but you're right, we'd be terrible if noone ever told us what we were doing wrong.

Posted by: Natalie Ferguson | Sep 18, 2006 3:05:48 PM

I would suggest that "white male fascist" conspiracy theorists are an atypical user group for most products AND that you would learn even more by observing users than mainly listening to people who complain.

I'd agree though that it's useful to be aware of what complaints there are out there.

Posted by: Jason Yip | Sep 18, 2006 3:28:20 PM

As much as I love the culture's positivity, the US is in so many walks of life hamstrung by it. Not to put too fine a point on it, people (such as astronauts) die because of it.

In order to improve, you have to Accentuate The Negative and yes that means even when you don't have an immediate solution for it.

http://makemarketinghistory.blogspot.com/2006/05/accentuate-negative-2.html

Posted by: John Dodds | Sep 18, 2006 6:03:28 PM

There are a couple of issues that your post has partially uncovered.
The first is in problems of user interface design - perhaps this is an unreasonable generalisation, but in my experience 9 out of 10 developers suck at user interface design. (and just before the finger pointing starts - I count myself one of those 9!)
But I don't think that it's our fault. Good developers are good developers partially because they don't think the same as other people. To some extent, Kathy's post about "Assumptions have a sell-by date" is an indication. Developers see things and work through logical problems in quite a different way to most people. While that works when things are being developed away from the public eye, it means that when those same 'different' thought processes get unleashed on a user interface, that regular users probably aren't going to get it.

The other issue raised is that interviews/surveys are hard! The fact that _you_ are interviewing the users is probably going to bias the bulk of the results in a meaningless direction right from the start. Add to that people's natural tendency to temper their criticisms, and you've really got a hard time of it.
One idea that would be interesting to explore is to use the phenomena that make call-in phone-poll surveys not work - and use it for good.
My understanding is that you can't gather any serious statistics from polls that require the user to call in themselves, because only the extremely passionate (either good or bad) will actually bother.
Given the difficulty that you've just described in trying to get critical feedback, if you can tap these passionate users at the time that their complaints are still fresh (and they're all fired up) you might get what you're after?

(criticism welcomed - this was a thought that occured to me in the last 5 minutes :-) )

Posted by: omni | Sep 18, 2006 6:05:07 PM

I agree with Omni that interviews with users are hard, as is good UI design. Our company is beginning version 2 of our product (internal users) and continually struggle with both issues. We still have users who absolutely dispise our version 1 product - even though we let the users help shape the UI themselves! Lesson number one: users don't always know what they want.

Since I'm better at UI than I am at coding, I handle most of the user calls and am usually at the receiving end of the screamers. It used to hurt my feelings that some people hated our product, but I've learned to listen and differentiate between the users who are frustrated just because it's not the system they're used to using, or if they have a true complaint that we can fix in the next cut. I calm them down and keep asking questions ("Really? I understand, so what could we do to make it easier for you to do that?")

In the past, many usability suggestions I made were fired down by developers saying that doing it my way would be hard or make really ugly code. But they are now seeing that we may have to do it the "hard" way in order to make it more usable. We want our users to get past the Suck Factor pretty quickly this time.

Please keep up the great posts, Kathy! Each entry I read either reminds me of something I've heard, or teaches me something new. You and everyone responding to your blog are more helpful than you'll know! :-))

Posted by: Lana B. | Sep 18, 2006 8:58:25 PM

Lana B. - absolutely true!
I'd forgotten that one... the other reason that developers are usually not good at UI work is that they are concentrating on a different game.
Developers are concentrating on the code and the underlying structure. An elegant design is one that fits together beautifully under the hood and is extensible and clean.
But (other than exceptional circumstances) the end user doesn't get to see any hints of the underlying design - so they don't (and shouldn't) care.

Posted by: omni | Sep 18, 2006 9:44:15 PM

I think we should listen to all the users but making a software that adapts to people is an utopia.
first : people are a lot more adaptable.
second : you can't know how everyone work (and it is in fact unpredictible)

Using efficient thinking there is only one solution : make software to which people adapt.
Even google search is not natural for a lot of people (at least at start).
The software should show the way to do things, not being able to do everything and at the same time inexplicably unable to do some needed simple things.

Of course we have to test the software, but the best software I know are the one that show the way, the one that ask us to adapt.
I don't know how to use VIM but I'm pretty sure that the users think it's the best thing since canned coke.
I use to draw on computer and I tried long ago a software absolutely unintuitive then gave up the thing, then used "everything you can do" software (like photoshop, or flash), but was never really satisfied because some of the needed basic drawing features are lacking, I don't need to have 2456 brushes, I need to have at least ONE beautiful one. People need that the software developer thinks and shows the path, not give them 365 possibilities thinking "there are so much any user could find the one for him".
That's a bit like if you are given 365 different color pencils but no basic grey one. This is a lack of thinking from the developer. Having so much makes people unable to adapt to the software because in fact... the is nothing consistent enough to adapt to. Humans need clear concepts to grasp the tool and make it theirs. When the tool is too complicated or diversified it burns all the brain power. Making a software full featured and masterable, that is the challenge. For drawing the pen is still the best, as well for developping ideas, where do the software fails?
well... just my two cents.

Posted by: isd | Sep 19, 2006 12:10:01 AM

Good point, but there is one trap we must be careful not to fall into, and that's the whole "you only learn from the dissatisfied users".

It's just as important to hear what works and what people like. Otherwise you may end up changing a feature 100 people like, the first time you hear from someone who hates that feature.

Also, it's crucial to be able to bask in the appreciation of the good work you've done :o)

In any given project we can learn equal amounts from what worked and what didn't work - and in most projects (hopefully) most of what we did worked, which is why there is more learning in the successful stuff than in the failed stuff.

Unfortunately we tend to pass over what worked very quickly...

Posted by: Alexander Kjerulf | Sep 19, 2006 1:10:07 AM

Couple of thoughts from this excellent post. First, "provably correct didn’t win the heart and mind of this user" is key. So many of us believe in "facts" ("it's [bigger, brighter, faster] than it was before; why are they complaining?") that we forget that facts don't really matter. Experience matters. If I experience the app as slower, then it's slower, no matter what the stopwatch says.

Second, this whole area points to the importance of "outliers." We love to say that "the vast majority of users feel..." Fact is, the ones out on the fringe (the types who call us "fascists" in moments of hyperbole!) tell us lots of great stuff...just as long as we don't write them off as "nutjobs." As Hugh McCleod might say, "listen to the wee voices."

Finally, Petroski's latest book is very relevant: "Success Through Failure: The Paradox of Design." It's very good.

Cheers!

Posted by: Tom Guarriello | Sep 19, 2006 5:31:37 AM

I would like to second the above comment. Be careful about changing your interface to accomodate a screamer. You may alienate a host of (semi-)satisfied users. I still use Visual Studio 6 as much as possible since the "wizards" in Visual Studio.Net take 3 times as long to do anything (but they are step by step). I gather that MS has seriously changed the user interface of Office 2007 for some reason as well.

Posted by: expr | Sep 19, 2006 5:32:25 AM

Developers do have habits of thinking that they can use to work around their UI blind spots, if they're willing to. They've already had to learn the habit of discarding fond theories in favour of evidence, because when something doesn't compile or a test fails, there's no fudging it. If they can get into the mindset of treating user feedback as being as true and objective as an error log, then developers can be quite determined and creative about fixing the issues presented, because they derive so much of their job satisfaction from being problem solvers.

Not to disagree with any of the above comments, but to present another example of an outlying case, recently a user phoned me with an issue, I thought about it, acknowledged that there was a problem, and was about to start discussing possibilities for solving the problem when instead I got a tirade about the flawed development process and how this user's department weren't consulted etc etc; and soon afterwards there was an email to various higher-ups about how not only was the system flawed, the developers themselves _admitted_ the flaw. I may have a UI blind spot, but I have a political blind spot that's ten times worse, and I had failed to recognise that this user wasn't phoning me with criticism, but was fishing for information to make trouble.

Posted by: Don | Sep 19, 2006 7:28:58 AM

I would suggest that "white male fascist" conspiracy theorists are an atypical user group for most products AND that you would learn even more by observing users than mainly listening to people who complain. (Jason Yip)

I'll lend my support to that idea as well. Unless this user has a rapport with you and intimately knows your sense of humour / political beliefs, I think the fact that she chose "white male fascist" (as if the first two words have anything to do with the 3rd), as opposed to something like "restrictive", says a great deal more about the user than your product.

Putting too much weight on criticism (especially the non-constructive kind) is heading in the direction of design-by-committee, and we all know where that leads. What may drive one user crazy could make 50 other users pee their pants. There's no substitute for actually observing users and seeing how they work with your product.

You're also comparing [eventually] specific criticism with vague praise or tacit acceptance. Specific praise would be just as valuable, but you're not very likely to get that under any circumstances, and the reason is one that you yourself have pointed out before: a good tool should be practically invisible, transparent. Users who find something very natural to use are not going to be able to point out specific things they like as easily as the dissatisfied users can point out things they dislike.

We should definitely listen to users, especially their criticism, but the key word is userS and not a user. I prefer to be scientific: observe several users, look for general trends, investigate the most common questions or complaints, and not necessarily assume that one user represents the whole community.

Posted by: Aaron G | Sep 19, 2006 8:09:30 AM

"Every product evolves. It’s the rare (or trivial) that gets it right the first time and sticks with it for the rest of time. Listen to the screamers and whiners and people writing nasty blog posts. "

Good post.

The guys at 37 Signals should give this a read. We had an interesting discussion recently related to this kind of thing.

http://darmano.typepad.com/logic_emotion/2006/09/does_basecamp_s.html

Posted by: David Armano | Sep 19, 2006 9:26:33 AM

Trackbacks on typepad seem to be down, so here's a very manual one :).

http://stupidapp.blogspot.com/2006/09/quick-links-round-3.html

Posted by: Guy | Sep 19, 2006 5:52:58 PM

The comments to this entry are closed.