« Interesting Moments | Main | Thank you and some Java book news »

The Quantum Mechanics of Users

Userpriorities

People have commented that "creating passionate users" means nothing more than "listening to users like we always have--DUH!" But if it were that simple, we'd all be producing--and using--products and services that people love. That meet real needs. That fulfill real desires. That help people kick-ass.

How, then, to explain the Grand Canyon-sized gap between what users really want and what we so often produce as a direct result of our sincere listening? Maybe the physics is wrong...

Light can behave as a wave, until you ask it to explain how it got from point A to point B, in which case it can behave as a particle. In other words, asking light to explain itself can change the very nature of how we perceive it. And this notion that sometimes "observing an event changes the event" comes up in many areas of quantum physics.

But it's not just the subatomic world that gets weird when you look too closely--in some cases, asking a user to explain his choices changes his choices! In his book Blink, Malcolm Gladwell (author of The Tipping Point) gives an example where students were asked to rank order 44 different kinds of strawberry jams. When compared with the rankings of experts, the students did fairly well -- "even those of us who aren't jam experts known good jam when we taste it." But--and here's where it gets weird--when the students were asked in advance to provide not just the rankings but a written explanation of their choices, the student rankings lost virtually all correlation with that of the experts. As Gladwell puts it, "By making people think about jam, [the researchers] turned them into jam idiots."

Think about that when you're asking for user feedback whether as focus groups, user questionnaires, or even usability testing (although the implications are different for each of these things).

So how can we hope to learn anything about what our users want and need if the very act of answering a question could change their answer? We have to get better at making inferences from what we observe without intervention. We have to get to the spirit of what we observe, rather than focusing on the specific details. We have to reconize that what they do says much more than what they say, especially when they're not saying anything at all.

Readers here left some great comments about this on my earlier Listening to users post:

Tim said:
The comments about listening to what the users are saying, what they're not saying, and how it's being said reminds me of the quote by Claude Debussy, "Music is the silence between the notes."

And Matthew Moran said:
It is not that we should not listen to clients/users but we should not let their limited understanding of what is possible, limit where the solution/software/project can go. It is important to listen and draw additional information into the open. In this way, we can discover what is truly desired but never contemplated from the client's perspective.

Paulo Eduardo Neves said:
Gilberto Gil, a great brazilian musician and the country current minister of culture, has a verse that says: "The people knows what they want, but the people also wants what they don't know".

Eric Stephens offered this link great post from Mark Hurst on Customer Research that includes:
"In our non-directed listening labs, we ask customers to use the Internet in the way they normally use it at home or work. While we do have a goal for the research, we try to let the customers lead us to the answer, rather than the other way around."

And Stu Max made it simple:
I guess that's how I'd wish you would reframe your point: You've always got to listen to your users, but sometimes you've got to listen beyond the words.

In addition to listening to users, we should observe them as a wildlife photographer or naturalist would--in the users native habitat, from a distance, with as little intervention as possible. We have to look for the whys based on the whats of their behavior. And when we do ask questions, the questions should be not just on specific behaviors ("why did you do it that way?") but also (perhaps more importantly) about what they value at an abstract level ("what does it mean to you to be using [whatever] in your [work/life]? How does it help you in (or prevent you from) kicking ass?")

This doesn't mean we shouldn't sweat the details--down to the last interface pixel, book font, metal finish, or drum beat. It all matters. And much of it can come from questioning users directly. The trouble is that this is where we tend to spend nearly all of our "listen to users" effort. We field complaints, solicit feedback, and accept customer requests. In other words, we focus on the trees and miss the forest.

Why not become "user naturalists" and find out what really makes our users inspired/frustrated/motivated/hopeless/passionate? Maybe the best way to find out what they need and want from our product (or from a future product we hope to develop) is by asking about something other than the product. Maybe they say they're satisfied with our product (or the category of products in which ours belongs), but we need to ask if they're satisfied with the very nature of what they're using our product for. Maybe asking about their favorite hobbies--the things they are passionate about--can help shine a light toward a new feature or capability (a new slider) we hadn't previously imagined. One that nobody ever associated with this type of product or service.

While we can still ask why they chose the blue button, we must understand that if we tell them in advance that they'll need to explain their choice, that knowledge could change the outcome. It might cause them to click the blue button when they would have clicked the green one! When you collapse the wave function, make sure that what you get is not simply what you caused by looking. : )

Posted by Kathy on December 21, 2005 | Permalink

TrackBack

TrackBack URL for this entry:
http://www.typepad.com/services/trackback/6a00d83451b44369e200d834253b0e53ef

Listed below are links to weblogs that reference The Quantum Mechanics of Users:

» http://www.associationinc.com/blog/index.php/224 from Association Inc.
Observing an Event Changes the Event: Physics works for organizations, too. We learn more from observing our members than from asking them to explain their choices. More here. No Tags... [Read More]

Tracked on Dec 22, 2005 7:11:01 AM

» Users Lie from ntschutta.com
I consider myself a client side guy. I arrived there rather by accident - I was one of the least experienced (in terms of years worked) members of a team that was slated to build the companys first Java based web application. When it came tim... [Read More]

Tracked on Dec 31, 2005 4:08:00 PM

Comments

"Schrodinger's Kittens" is a good book that deals with the waveform/particle duality of light and the wacky phenomenon of "the act of observation changes the outcome." It gets weirder than you think- if you dig this type of stuff Kathy check out "The Holographic Universe" too for another mind-blowing read.

You are right on with this analogy as it relates to querying users. I don't remember the strawberry jam example from Blink but the one he gave that really resonated with me was the experiment in which they asked college students to pick a poster they wanted for their dorm room. Almost everyone opted for abstract art. Then the experimenters conducted the same exercise only they asked the students to explain their choice. The result was that nearly everyone ended up picking these lame posters with kittens and rainbows and crap- the kind of stuff you find in a doctor's office. Their justifications were responses like "because they're cute and remind me of this time when blah blah" (the theory is that they were subconsciously contriving those answers in realtime to apparently gain the approval of the experimenter). And the true test then came with the satisfaction poll they conducted later that year (because the students actually had to live with the poster they chose). The ones that had to explain their choice ended up _hating_ their posters. The minute you try to dissect a gut decision you've thwarted the "thinslicing" from occurring and disarm your instinct. Consistent with this research, using non-interfering usability studies would seem to promise the best means to get true feedback on software and web apps.

good stuff kathy.

sean

Posted by: Sean Tierney | Dec 21, 2005 12:26:46 PM

I remember an exercise that swatch did when they first came out; organising a display in a hotel foyer, then picking people off the street to ask which watch design they'd choose if they were getting one that day. The market researcher duly noted the persons choice.

On the way out, they were asked to "just take one watch off the other stand on the way out to keep as a thankyou for participating". Which all did.

The two stands were identical. However, the watches chosen for the research and which were taken were completely different...

I think it's "Say's law" that suggests you'll never know the facts until you ship something. Market research and people have a habit of not mixing too well!

Ian W.

Posted by: Ian Waring | Dec 21, 2005 1:31:44 PM

Kathy,

One reason I'm so big on the use of personas, scenarios and storyboarding in Web design is that I see and feel something "missing" in the process of _only_ observing, asking, listening and thinking about what users do.

C.S. Lewis (of Narnia fame) wrote something very interesting in his essay, "Myth Became Fact" (found in his collection, _God in the Dock_. Forgive the long quote because he says it so much better than I could:

"Human intellect is incurably abstract... Yet the only realities we experience are concrete--this pain, this pleasure, this dog, this man. While we are loving the man, bearing the pain, enjoying the pleasure, we are not intellectually apprehending Pleasure, Pain or Personality. When we begin to do so, on the other hand, the concrete realities sink to the level of mere instances or examples: we are no longer dealing with them, but with that which they exemplify. This is our dilemma--either to taste and not to know or to know and not to taste--or, more strictly, to lack one kind of knowledge because we are in an experience or to lack another kind because we are outside it. As thinkers, we are cut off from what we think about; as tasting, touching, willing, loving, hating, we do not clearly understand. The more lucidly we think, the more we are cut off; the more deeply we enter into reality, the less we can think. You cannot _study_ Pleasure in the moment of the nuptial embrace, nor repentance while repenting, nor analyse the nature of humour while roaring with laughter...

"Of this tragic dilemma, myth is the partial solution. In the enjoyment of a great myth, we come nearest to understanding as a concrete what can otherwise be understood only as an abstraction."

Whew. Well, I take this a bit further than Lewis. Given that our minds are wired for story, I believe stories can be a bridge between knowledge and experience.

While watching, asking, listening to our users, we are removed from their experience. Lewis might say we're in the world of abstract thought and thus cannot fully understand what the user is experiencing.

My "partial solution" are personas, scenarios and stories based upon the findings of my user analysis. I obsessively and dutifully gather all I can directly from users and let it stew in my mind. At some point, imagination kicks in. The abstract data begins to emerge in narratives that can take the form of personas, scenarios or stories about the user experience of the project I'm working on.

As I work these stories out, they help me "enter into" the users' experiences more fully than I could by producing a mere report or PowerPoint of my findings. And stories are more powerful ways to communicate those findings to the project team.

Dave

Posted by: Dave Rogers | Dec 21, 2005 7:35:54 PM

A couple of these thoughts are echoed in agile programming - the idea of user stories, and the reality of getting *something* no matter how small, but which is of some use, into their hands - because that's when what they REALLY want starts to come out, and also when what they didn't know they wanted is teased out.

Posted by: Ric | Dec 22, 2005 5:00:46 AM

Great points (as always).

What you are describing is termed ethnography by anthropoligists.

Ethnography is a method of studying and learning about a person or group of people. Typically, ethnography involves the study of a small group of subjects in their own environment.

Rather than looking at a small set of variables and a large number of subjects ("the big picture"), the ethnographer attempts to get a detailed understanding of the circumstances of the few subjects being studied.

Ethnographic accounts, then, are both descriptive and interpretive; descriptive, because detail is so crucial, and interpretive, because the ethnographer must determine the significance of what she observes without gathering broad, statistical information.

Clifford Geertz, whose thoughts about culture are excerpted in the Other Important Definitions of Culture, is famous for coining the term "thick description" in discussing the methodology of the ethnographer.

Definition above courtesy of Washington State University - Virtual Campus - Learning Commons glossary: http://www.wsu.edu:8001/vcwsu/commons/topics/culture/glossary/ethnography.html

I'm finding that my anthropological knowlege is actually benefiting my IT carreer -- after all anthropology is the study of humans, and users are human. It somtimes helps me to fill in the gap between what users say they want, and what they actually need to accomplish their goals.

Ric's comment about "teasing out" what users didn't know they wanted is right on target. To me, it's analogous to unpicking embroidery - slow careful examination of the puzzle - but in reverse! LOL

- Karen

Posted by: Karen | Dec 22, 2005 10:42:23 AM

Amazing. Not even a hint of the word 'empathy'. Is this really about creating passionate users? By observing them like some weird wildlife species? I thought we've already made some progress in that field (like, putting yourself in your user's shoes). But no, no mention of any empathy.

It's a bleedin' shame!

Posted by: Alex Bunardzic | Dec 22, 2005 10:50:14 AM

It is interesting to me that although we couch our verbiage in scientific analysis and the Graeco-Romans used allegorical language that many times the message is the same as it has been for centuries.

In the legend of Eros (Cupid) and Psyche - Everything is fine with Psyche's marriage until she decides to examine her husband. While Eros represents unthinking desire - Psyche represents objective analysis. When Psyche attempts to use objective analysis to examine unthinking desire (allegorically she lights an oil lamp to gaze upon her husband) - unthinking desire is lost through examination.

As long as the students were asked which Strawberry Jam they prefered the answers they gave reflected their Desire without Analysis (Eros) but as soon as students were asked to apply Psyche (objective analysis), Desire (eros) is lost.

Bulfinch's rendering of the Psyche and Eros story is available at: http://www.bulfinch.org/fables/bull11.html

Posted by: James Shewmaker | Dec 22, 2005 11:40:45 AM

I don't know how you continue to keep producing such super content.

This discussion reminded me of a brief set of guidelines my partner wrote for conducting surveys. It boils down to asking people what they've DONE vs asking them what they'd do? It's never the same thing.

The guidelines are available among the freebies at http://www.wizardacademypress.com/goodies.asp Just scroll about 2/3rds of the way down the page and look for the guy with the binoculars.

Keep up the great work!

Posted by: Dave Young | Dec 22, 2005 2:36:32 PM

I have to agree with Dave about the "superb content".

The problem you describe, Kathy, relates to why it is so difficult for executives to appreciate the value of market research. Since most people are themselves consumers, they think they are already experts on the market. Since anyone can listen to customers, most people consider themselves qualified market researchers.

Yet listening to customers is not enough. Asking questions isn't enough. You have to ask the right questions in the right way. Often the questions you ask aren't the questions you ultimately want answered (see http://cauvin.blogspot.com/2005/08/framing-survey-questions.html). And in some cases you're better off with the ethnographic approach you mention, in which you observe without disturbing or asking any questions.

Posted by: Roger L. Cauvin | Dec 22, 2005 3:38:25 PM

the thing is that no one really knows what want. that takes way too much time and introspection to figure out. so what people do is they look around and see what everyone else is doing and wanting and then use that instead because it feels like something they would want too. basically desire is appropriated instead of created because its just too damn hard otherwise.

this is why there are some people who look deep into the structure and usability of things in order to come up with things to make life easier on the folks that have better things to do and cant be bothered.

a big problem comes when you look so deep that you forget that other people do not. so the things that you might find valuable in the depths end up being extraneous to those on the surface because your point of view is skewed.

the hardest thing is to bring both of these perspectives together in a meaningful productive way.

Posted by: markus | Dec 22, 2005 8:26:14 PM

I've had quite pleased users through the give and take of an iterative design process. But I've only had screamingly orgasmic users when giving them what they've asked others for repeatedly.

Posted by: a | Dec 23, 2005 7:16:58 AM

This discussion reminds me of the book WHY WE BUY, by Paco Underhill, who scientifically observes customers in stores.

Posted by: David | Dec 23, 2005 8:52:13 AM

Hey Kathy, have you been NSA-ing my email? The title of the J1 BoF proposal I submitted last month is Computer Books and Quantum Gravity. If you haven't read Lisa Randall's, Warped Passages: Unraveling the Mysteries of the Universe's Hidden Dimensions, you should check it out.
http://www.harpercollins.com/global_scripts/product_catalog/book_xml.asp?isbn=0060531088
btw--Happy First Anniversary. Great blog!

Posted by: Greg | Dec 29, 2005 9:41:40 AM

Hi,

Very interesting article, intriguing comprison with the duality principle:)
Good lesson for me, that one should be very careful when asking questions...

--
www.e-verdict.com
People's Opinions and Judgments

Posted by: e-verdict.com | Jan 8, 2006 5:44:37 AM

Kathy: this is a book you need to write. Now.

Posted by: Keith | Jan 14, 2006 5:48:04 PM

The comments to this entry are closed.