« Sensemaking 1 | Main | Sensemaking 3:The search for a representation »

Sensemaking 2: What I do to make sense

Rugby_scrum

I’m a professional research scientist. Research is what I do day in, day out. So what I do for sensemaking might not be what you do at all, or it might be really relevant. I don’t know.

But I do know that sensemaking is a big, big part of my job. And I know that there are sensemaking methods…and then there’s what actually happens. I’m going to ignore the sensemaking prescriptions for the moment and focus on what really goes on.

The common conception of research is that a scientist first thinks up a hypothesis, then collects data to test it, then writes up a neat analysis confirming or disconfirming the hypothesis. That’s beautiful, but it’s also almost completely wrong.

When I’m making sense of some complicated area, it’s more of a full-contact, sweaty, wrestling-around-with-data kind of thing. Trust me: it’s not nearly as antiseptic and passionless as the common conception would have it. This is red-blooded science as played on the field. It’s more of a rugby scrum than a still-life chess game. Here’s how it goes for me…


FIRST: Figure out what it is that you’re trying to understand or get done—let’s call this the domain. This is crucial because you can waste a lot of time doing various experiments or studies on the wrong thing. I find it valuable to write out a quick summary of what I’m trying to understand… often just as a note in my notebook, or maybe in an email to a friend. The point is that you need to frame things because the act of framing helps to focus on what to do next.

Of course, it’s not always so neat. Sometimes you only come to realize later what the topic really is. Sometimes you frame a domain one way and later discover that this isn’t a good framing at all. Then things shift and that’s where it gets interesting.  And then every so often you end up by accident in a domain that you hadn’t planned on studying. That’s serendipity, and it happens more than you’d think. Regardless, knowing your domain leads to the next step…

SECOND: Collect a lot of information about the domain. I like to read about how others think about the domain, what works and (just as importantly) what doesn’t work. Sometimes people will say how they think about the whole domain, and that’s often incredibly useful because they’ll have already organized things in a way that’s useful. I look for varying opinions and results. In the discrepancies between accounts there’s often something of interest. (Something like this: “Smith says people always do X, but Jones says they never do X… how can they both be right?”)

I should point out that often you can make sense of something just by doing this kind of research. It happens all the time—you’re working on a domain, and then someone writes an article that explains it all. A newspaper reporter would call that “being scooped.”

As you collect and organize, you'll find that you can target your collecting more finely.  That's when I'll start to do experiments (to collect exactly the data I need). 

THIRD: Organize the information.  Depending on what the domain is, you might look for an organization that helps you see the entire structure of what’s known, or you might end up building a very detailed model. In almost every case, I spend a lot of time figuring out how to organize the information I have into some kind of representation. This representation could be something as simple as a bunch of folders with topics, or it could be a much more complex spreadsheet model. But figuring out how to represent (and organize) what I know is key. Making this representation is usually when the “ah-ha” moments happen. It’s when pieces of the puzzle start falling into place and I realize the connections between everything.

FOURTH: Iterate. Realize that you almost never get it right the first time. Sometimes I’ll get the original domain wrong, and I’ll be studying the wrong thing. Sometimes I’ll collect a lot of junk information that I have to winnow out. And sometimes I’ll just create representations of that information that don’t work out.

In any case, keep iterating—on the domain, on the information set, on the representation—until you’re to a point where you can satisfy your need to make sense of the original domain.

FIFTH: Do. As in, do whatever it was you wanted when you figured out what the domain was. Research of the kind I work on is always trying to figure out something so you can do the next thing.

I know this is all a little abstract, but tomorrow I’ll post with a real example of a research sensemaking story that will map onto all of these steps. Then Friday I’ll summarize all of the amazing and remarkable comments folks have been making on the “Sensemaking 1” post. (Thanks for all your comments, btw, I’ve been enjoying reading them!)

Bydan_2

Posted by Dan Russell on January 24, 2007 | Permalink

TrackBack

TrackBack URL for this entry:
http://www.typepad.com/services/trackback/6a00d83451b44369e200d8342de10853ef

Listed below are links to weblogs that reference Sensemaking 2: What I do to make sense:

» Sensemaking how-to from Planblog
Dan Russell over at Creating Passionate Users has a nice article (part two of a three-part series): Sensemaking 2: What I do to make sense His personal process is as follows: FIRST: Figure out what it is that you’re trying to understand or get done... [Read More]

Tracked on Jan 24, 2007 12:49:28 PM

Comments

Mr Russell, Nice pointers for the go. Though, what I may be interested is pointers or your experience in the following.. When I start looking around for new study topic eg:tutorials, the quest for new topics seems never ending..meaning, I am stuck on gathering material, stuck on finding the best tutorial there is and it happens for all the things, that I have to work on. Even for a book, I go overboard thinking about paperback vs hardcover, going to extent to checking the paperback in barnes&noble, and then ordering it online. If I have to pick an UML tool, I go all over and search for the perfect one..starting with the design concepts implementation,perfect visualization to something cynical as the perfect font..
Perhaps, if you have not already labeled a medical paranoia onto me,I will appreciate you writing from experiences from early days, if you have undergone similar stuck-on-gathering/not-enough material phases.

Posted by: man | Jan 24, 2007 10:08:08 AM

I noticed this is pretty much exactly how the microformats process does sensemaking in the context of semantic HTML publishing:

1) Why? = Figure out what it is that you’re trying to understand or get done
2) Document Current Behavior = Collect a lot of information about the domain
3) Propose a Microformat = Organize the information
4) Iterate = Iterate

Posted by: Scott Reynen | Jan 24, 2007 10:09:13 AM

What's your research on Dan?

Posted by: hmm | Jan 24, 2007 11:49:42 AM

One of my buddies emailed his managers the Sensemaking 1 post and I'm pretty sure we all got here for SM2. The first thing that struck me was the relationship with Torbert's seven stages of leadership (Harvard Business review, 4/05, "7 Transformations of Leadership") in which after 30 years of research we know that there is an hierarchy of ability to think in ways that enable one to "lead" in better or worse ways, and a valid, reliable instrument (hard-to-cheat-on questionnaire) that measures where you are - more correctly - the spread and centroid of your answers across those 7 stages.
I took the only training in how to grow people up the stages offered in this country in 10/05, and the (very few) techniques are very much about sensemaking. Based in Chris Argyris' and Russell Ackoff's Systems Thinking work as well:
1. frame the problem to solve (bound / describe the domain?)
2. illustrate the problem, context and/or options ("get sweaty and dirty" trying to apply the concepts to known or new situations to learn by doing how "it" works)
3. advocate a solution alternative
4. inquire (and iterate).
It turns out that the dividing line between people that you should allow into management positions is between stages 3 and 4, and the defining criteria is "self-oriented" thinking below 4 and "others-oriented" thinking 4 and above, while 55% of people 1-3 and only 45% in 4+. I would expect your responses to be along a scale of thoughtfulness (...) that mirrors the Leadership Development Framework (LDF) scale.

For myself - I find that sensemaking is a result of experiencing that "cognitive dissonance has occurred" - what do I do now?
1. Make sense now or later? = Impact analysis. Example - magic tricks - person appears to move through a solid plate glass window - answer = not now or later. Example - person at work is disruptive: long social visits. Answer = not now, but before later. Example - customer or supplier reneges unexpectedly on supposed commitment. Answer = now.

2. Verify facts - observations by multiple stakeholders requested, especially primary actors or their confidants. If extreme variation, try to replicate event / condition with some other or same observers. Guage impact on stakeholders.

3. Build mental model - what was expected (by me and by other stakeholders)? what happened? did context / environmental factors drive the unexpected result? what are the causal factors? Is it possible or useful to be able to manipulate or influence this result from happening in the future? If not possible, what's the risk - expected frequency of the event, how negative are consequences for me / others? If possible, who is the logical agent to perform the influence / manipulate the situation? Consider multiple "criteria for success" about event from stakeholders, not just mine.

4. If useful to me to be "sure" - design and conduct "experiment" to see if hypothesized influence / manipulations work. If so - great, if not. Repeat 1-3.

5. If apparently useful to me or them, communicate hypotheses (mental models) to affected stakeholders (or subset) and request feedback (iterate). If useful to me, pursue a cycle or two. If for others, participate only if they indicate value in having me continue to participate.

What do you think?

Posted by: Pete Malpass | Jan 24, 2007 1:18:40 PM

That sounds really good and in step 4 Iterate we need to add some questions to make sure we are making progress. We really don't want to get stuck in analysis paralysis. we have to make some decisions to help us get to the next level.
The reason I like trying to find experts in a domain is that with that name I can search and find the opposition and what they have to say. They almost always reference each other by name so that makes it easier. The data they provide contradicting each other is the most valuable bit and allows me to see what it is they are talking about and make my own conclusions.

Sensemaking is good stuff and like usability needs some innovation to find a good way to actually measure what you think you are looking at.

Posted by: Stephan F | Jan 24, 2007 10:53:30 PM

I confess, most of your post went phwoop! over my head (I'm obviously wa-ay out of my league), but I felt I wanted to applaud your choice of photo and the brief analogy. Rugby. Now there's a sport! Especially the Union variety

Posted by: Karyn Romeis | Jan 25, 2007 2:12:56 AM

Figuring out that the domain was wrong all along is probably one of the most satisfying feelings one can get. It opens up new avenues that feel like you've reached a new level of consciousness.

Posted by: Vorlath | Jan 25, 2007 3:34:35 AM

Nice job! I enjoyed reading this article, nice tips!

Posted by: Anthony C | Feb 8, 2007 5:59:28 AM

I am stuck on gathering material, stuck on finding the best tutorial there is and it happens for all the things, that I have to work on. Even for a book, I go overboard thinking about paperback vs hardcover, going to extent to checking the paperback in barnes&noble, and then ordering it online. If I have to pick an UML tool, I go all over and search for the perfect one..starting with the design concepts implementation,perfect visualization to something cynical as the perfect font..

Posted by: Bank zdjec | Aug 18, 2007 3:54:03 AM

The comments to this entry are closed.