« My First Scoble: a DIY gift idea | Main | Five(ish) Things I Don't Know About You »

Don't make the Demo look Done

Managingexpectations

When we show a work-in-progress (like an alpha release) to the public, press, a client, or boss... we're setting their expectations. And we can do it one of three ways: dazzle them with a polished mock-up, show them something that matches the reality of the project status, or stress them out by showing almost nothing and asking them to take it "on faith" that you're on track.

The bottom line:
How 'done' something looks should match how 'done' something is.

Every software developer has experienced this many times in their career. But desktop publishing tools lead to the same headache for tech writers--if you show someone a rough draft that's perfectly fonted and formatted, they see it as more done than you'd like. We need a match between where we are and where others perceive we are.

Joel Spolsky talked about this way back when in The Iceberg Secret, Revealed. The secret:

"You know how an iceberg is 90% underwater? Well, most software is like that too -- there's a pretty user interface that takes about 10% of the work, and then 90% of the programming work is under the covers... That's not the secret.
The secret is that People Who Aren't Programmers Do Not Understand This."

He goes on to add corollaries including:
"If you show a nonprogrammer a screen which has a user interface that is 90% worse, they will think that the program is 90% worse."
and
"If you show a nonprogrammer a screen which has a user interface which is 100% beautiful, they will think the program is almost done."
You'll have to read the rest to get the other corollaries, and to see where else he takes the topic.

My First Robert Scoble talked about this recently, in a story about the early fake prototypes of Vista:

"Later... I found out that all we really saw were Macromedia Director-based movies. They looked so cool...how good they made us feel... This actually was NOT a good thing for Microsoft...when you build up expectations and you aren't able to meet them you look pretty silly.

But behind the scenes things were even worse. Why? Because executives bought into the Flash and Mirrors song and dance too. They thought what they were seeing was possible... it's very possible that what you are dreaming of is simply not possible."

So, overpromising by delivering a flashy (or Photoshopy or Powerpointy or Visioy) demo is tempting, but it's short-term gain (you're a hero to your client, boss, the public) with long-term pain. But there's another problem with overdone demos that's just if not more damaging than wrong expectations:

The more "done" something appears, the more narrow and incremental the feedback

Feedbackimage

We see this with books and software all the time. Show them something polished and pretty, and you'll get feedback on font sizes. The reviewers make incremental tweaks, blinded by what's in front of them. But show a napkin sketch, and they don't just see what's there, they see what's possible. Obviously you need to tell reviewers about the kind of feedback you DO want at this stage... you don't want big-picture-forest feedback when you've really moved on to the tree stage. My point is: all you'll get is tree-tweaks when you show something finished-looking, so if you want big picture, make it fuzzy!

Finally, it's great to know that there are tools to help make the look match the state, with my favorite being the Napkin Look and Feel, a GUI "skin" for Java that makes the interface look--quite literally--like it was scrawled on a napkin. I think it's brilliant, and creator Ken Arnold (Java guru, fellow Sun refugee) paid astonishing attention to detail. For example, if the radio buttons were all exactly the same, no matter how sketched they look, their exact sameness would break the spell. So, there's more than one of nearly all of the GUI components from sliders to buttons to tabs to...

Here's just one picture, but I urge you to go check out the snapshots on Sourceforge or even better, try the actual Java demo (you can get to the demo from the link above).

Napkintoolbar

I realize that there are a million exceptions and caveats (like, for example, when you'll be fired if you don't show something jaw-dropping early on), but in general, the more closely what you SHOW matches what you HAVE, the more likely you are to have less pissed-off people down the road, and the more likely you are to get much better feedback, at the stage you need it. Bottom line: when it's an early demo, think fuzzy. Think sketchy. Think underpromise-and-overdeliver.

[Related links:
* 37Signals blog talks about how to make sure you fix the 'placeholder' stuff before the final release
UPDATE: flow/state discusses the same thing in Matching Design sketches to the desired level of design feedback]

Posted by Kathy on December 27, 2006 | Permalink

TrackBack

TrackBack URL for this entry:
http://www.typepad.com/services/trackback/6a00d83451b44369e200d83462751469e2

Listed below are links to weblogs that reference Don't make the Demo look Done:

» Managing expectations from HMK's Spurious Thoughts
Another great post from Creating Passionate Users: Don't make the Demo look Done. But you have them in your reading list already, don't you?... [Read More]

Tracked on Dec 28, 2006 2:25:37 AM

» Napkin Look Feel for Java Prototypes from Jim Harte's Blog
Java Swing is often criticized for its lack of native widgets. Controls in Swing are rendered by Java code, rather than the underlying OS. This can create problems like having a Java app that doesnt have the same look feel... [Read More]

Tracked on Dec 28, 2006 8:07:31 AM

» Don't Make your Demos Look "Done" from plugim.com
A great concept of how polished your product demo should look when showcasing it to possible vendors, investors, users, etc. [Read More]

Tracked on Dec 28, 2006 11:36:58 AM

» Sooner or Later? When Does a Client Get to See Their Show? from Thomas R. Clifford
73302076_427e6220f5_m... <a href=[Read More]

Tracked on Dec 28, 2006 11:42:11 AM

» How to not suffer with Works in progress from Design Sojourn
Designers always suffer with managing and showing their design works in progress". The problem is usually designers don’t get the feedback they want, or do too much and miss the dateline, or they do too little and the client feels they are no... [Read More]

Tracked on Dec 28, 2006 8:07:41 PM

» Managing Expectations from Ponder Marketing
So youre ready to make the big presentation for a new product, a new campaign, a new venture. Maybe youre seeking feedback, or maybe its just a status report. The natural instinct is to make things look as good... [Read More]

Tracked on Dec 28, 2006 9:04:15 PM

» GNC-2006-12-29 #228 from Geek News Central
Last show of the year so I do some wrap up stats and of course bring you a full line of tech with a special performance at the end of the show by my two aspiring artist here in the... [Read More]

Tracked on Dec 29, 2006 2:19:21 AM

» Daily Links: Sarbox, Managing Demo Expectations, Slow adoption for Vista, iTubes Clogged, IT expertise, VoIP hacking, Muni-broadband, Asian Quake from Network Performance Daily
NPD: A critical look at Sarbanes-OxleyPassionate: Don't make the demo look done.Security.ITworld.com: After one month, no rush to adopt Vista. CNN Technology: Music denied -- shoppers overwhelm iTunes.Enterprise Architecture: Thought Leadership: How to... [Read More]

Tracked on Dec 29, 2006 10:21:26 AM

» Why Napkin Drawings are Better than Finished Demos from Outside Innovation
Kathy Sierra has a great post on Creating Passionate Users, entitled Don't Make the Demo Look Done! Her topic is ostensibly the perils of showing demos of software with polished user interfaces. There are two significant downsides that she describes: [Read More]

Tracked on Dec 31, 2006 8:02:02 AM

» Don't make the Demo look Done from Lee Hopkins: Better Communication Results - business communication, online, blog and podcast consultant
The sublime Kathy Sierra argues for keeping the 'flash, puff and magic' out of presentations until the final version is ready for public presentation. In a post that is largely about computer coding and product design, the points Kathy eloquently makes... [Read More]

Tracked on Jan 1, 2007 6:55:30 PM

» Hobo : plugin Rails pour prototyper encore plus vite... from Denis au fil du web
Au cours d'un chat avec François Lamotte celui-ci m'a aiguillé vers Hobo un plugin pour Ruby on Rails, le framework pour le web qui offre une rapidité de prototypage inégalée. Hobo permet d'aller encore plus vite. Un screencast de 10 minutes en... [Read More]

Tracked on Jan 2, 2007 4:47:20 AM

» Wie man besseres Feedback bekommt from M-Block
Sie haben da einen Kommafehler im Text! Zwischen Lorem und Ipsum Feedback das vom Kunden kommt ist nicht immer brauchbar. Schließlich interessiert niemanden die Schriftart, wenn man noch nicht einmal die Funktion der Seite besproc... [Read More]

Tracked on Jan 3, 2007 4:45:17 PM

» Demo Days from Always
Kathy Sierra has an excellent post on the problems with showing a stakeholder a demo. Do you work your ass off and make the slickest demo possible or do you show them a sketch on a napkin? The thing I... [Read More]

Tracked on Jan 12, 2007 7:23:45 PM

» Managing Customers Expectations - summary from KDS Software Group
Posts reviewed: Dont make the Demo look Done by The Creating Passionate Users blog The Iceberg Secret, Revealed by Joel Spolsky The bottom line: How done something looks should match how done something is.... [Read More]

Tracked on Jan 14, 2007 9:22:29 AM

» Links of interest from Confluence: EB2 Technology
Kathy Sierra's great blog post 'Don't make the Demo look Done' [Read More]

Tracked on Jan 14, 2007 10:11:23 PM

» Showing-and-telling a marketingartform from Alive Matters
Creating Passionate Users, fast becoming one of my favorite blogs, shares some practical thoughts in this article about demonstrating new products or technology. Dont Make the Demo Look Done applies directly to software, but the i... [Read More]

Tracked on Feb 1, 2007 10:43:42 PM

» Soma. from Soma without rx.
Soma. [Read More]

Tracked on Sep 21, 2009 12:30:45 PM

Comments

Good advice! Reminds me of the flow|state blog article about the same subject:

http://miksovsky.blogs.com/flowstate/2006/10/using_crude_ske.html

Posted by: Alistair | Dec 27, 2006 9:13:50 PM

I only recently started reading your blog after a fellow Lotus Notes blogger did a trackback to one of your articles. A lot of what you point out are things that I have gut feelings about, but you back it up with concise expertise and clear examples of *why*.

Thanks for sharing, it gives me a lot of good information to share with my boss. It's just hard not to say "see, I TOLD you so!"

Posted by: Charles Robinson | Dec 27, 2006 9:52:36 PM

How the recipients of a demo react also depends on what their expectations are coming in. No one to whom you're demoing walks in as a blank slate. Complicating matters further are those, like my wife, who always assume that what you show them is a finished product -- even when you've stressed firmly that this is NOT A FINISHED PRODUCT.

Posted by: John Windsor | Dec 27, 2006 10:24:06 PM

Good points. I think everyone who has ever given a demo has experienced this. For websites I sometimes create un-styled, colorless HTML-prototypes to be able to get the structure of the site correct. Being able to click through "real" pages and seeing how the navigation works can be very useful and does more then 20 pages of text, trying to explain the structure in words.

However, even when you tell people multiple times the demo is about the structure, is bare-bones, and NOT about the design, you get responses like "shouldn't we add some color and pictures?" :)

Posted by: matthijs | Dec 27, 2006 11:52:03 PM

Yes, the "iceberg secret" is a real headache for software developers.

You're right on in terms of avoiding the perception of *being* done by not *looking* done.

I think our brains are just hard-wired to sharpen fuzziness (sketches, scribbles) by imagining missing details, and to fuzz exactness (clean, crisp mockups) by tweaking it to add our stamp (font, color, layout changes).

A technique I often use when presenting ideas (especially with remote team members) is to scribble the different screens on a whiteboard, then take digital photos. The images can then be assembled into interactive HTML mockups for pretty good low-fidelity prototypes or for use in the story board conversations you talked about.

Posted by: Dave Churchville | Dec 27, 2006 11:53:24 PM

Mr. Windsor's got a point: saying that it's unfinished usually has its meaning evaporate as soon as the reviewer starts doing their thing, whether it's pointing out how crappy your fonts are, pointing out that your new ice cream flavour needs more salt, or contorting your semi-built bird house to see if it's strong enough.

If you turn Kathy's argument around and approach it from the view of the reviewer, this makes sense: it's easier to critique the little things that I can already see, just because it doesn't tax my brain as much; pushing my imagination to ask why fundamental things in the work exist or don't takes a lot more thinking if the little things are already apparent.

N.B. that you'll get a more focused, "tighten this and that" set of suggestions from someone who has an idea of what you're doing (because coming up with those "little thing" suggestions is easy/immediately apparent) than someone who doesn't (e.g. a curious 7-year old asking his dad about his computer program).

Posted by: Edward Ocampo-Gooding | Dec 28, 2006 1:00:10 AM

The 'napkin' stage is also very useful in showing stakeholders where the unknowns lie and getting them to understand why their time might disappear before all the features they want are implemented.

I find that most stakeholders in any project don't understand how much anything unknown can eat up time. An 'unknown' being a new aspect you've never included before such as streamed video with synchronised events.

They assume because this feature is available elsewhere on other products, that you should not only be able to clone it, but customise it, too. Even if you've never coded/ designed anything like it before.

If you hide these unknowns, the stakeholders are only ever disappointed when you deliver an 80% complete version. Their disappointment seems unfair since you know how hard it was to get the 80% together.

Posted by: Simon | Dec 28, 2006 1:05:38 AM

Great post and one that rings very true to me. From my own experiences I'd add that this applies to internal clients as well as more traditional customers.

Recently a programme maanger and myself were sneaking round the office like kids in a playground trying to demo off a new feature of our application without the director seeing it. The demo looked useful and flash and customers are clamouring for it to be added to te app but even though it wasn't finished we knew that the minute he saw it it would be on sale.

We almost succeeded. We bought ourselves a couple of weeks at any rate!

Posted by: James Randall | Dec 28, 2006 3:52:04 AM

This is good advice for web designers showing a comp to a client also. Having too finished a look encourages the client to drive you crazy with change requests for different colors, fonts, etc., when a less finished looking version might not turn normal client into a client from hell.

Posted by: Virginia | Dec 28, 2006 5:59:39 AM

Excellent article. I have suffered even worse when I gave out a demo to a client (a piece of software that was unfinished) and the following week, I went to show them another version and I noticed they had started using it for production, loading real data... Finally I had to figure out a way to preserve that data in subsequent demos!! That was a really headache!!
By the way, I don't understand very well what Kathy means by "incremental" changes.

Posted by: Marcelo Ruiz | Dec 28, 2006 6:02:12 AM

Great article, but you almost just talked about showing the clients something too polished. I've just read an article about the other side of the coin. Here a quote:"...three years ago, while working on an on-line casino project. We developed much faster than we initially estimated, and the software was ready for delivery before the layout design was finished. We decided to deliver the software early, hoping to make a good impression on the clients, but the exact opposite happened - clients hated the mock-up layouts so much that they wanted to cancel the entire project."

Posted by: Paulo Neves | Dec 28, 2006 7:34:12 AM

Brilliant. I read this and first went "duh", then went "so why don't I do it?" I just recently started using lorem ispum dummy text on all unfinished pages, but some of the ideas linked to by the article/comments are great (using temporary CSS classes to give unfinished elements a hot pink background with yellow text - so simple and yet so effective!).

Now if only there were a way to survive those final "pre-launch" meetings where you have 10 people sitting in a boardroom who spend 2 hours arguing about whether a heading should say "My Account" or "Your Account" or "Account Information"...

Posted by: Aaron G | Dec 28, 2006 8:00:29 AM

Your blog is driving me nuts!

Every time my RSS reader informs me that you've posted a new item, I cringe. "How many good ideas does this woman HAVE?" I wonder. "Is it even statistically possible for her to have written yet another outstanding post?" Amazingly, the answer to the last question is almost always, "Yes!"

If I turn into a Passionate User addict, Kathy, I'm holding you personally responsible!

Posted by: Paul P. | Dec 28, 2006 9:14:28 AM

Replace everywhere you wrote "software developer" and "non-programer" with "physics brainiac" and "military brass" and you'll have a similar story. Sigh.

Posted by: Cyndi L | Dec 28, 2006 10:14:04 AM

Dave Churchville: I love that idea of taking photos of whiteboard sketches. I've done that in other contexts -- like to make quick handouts -- but I never thought about how useful that would be for making presentations like this.

Edward: You triggered another notion I hadn't thought about when I wrote this -- that people often feel the need to come up with something to critique when asked, even if under other circumstances they might have no feedback. We've all had the "that person's making a point just to make a point..."

James Randall: yes! I've so been there... especially on game teams where we just couldn't help ourselves (especially when sleep-deprived) from throwing random potentially cool things in with no thought to whether they'd be appropriate or even do-able. We did it partly to keep motivated and amused (sometimes just to entertain one another), but heaven forbid a marketing person get a look or--you're right--next thing you know they've called a press conference ; )

Virginia: You gave the real advice: "a less finished looking version might not turn normal client into a client from hell."

Aaron G: Now if only there were a way to survive those final "pre-launch" meetings where you have 10 people sitting in a boardroom who spend 2 hours arguing about whether a heading should say "My Account" or "Your Account" or "Account Information"...

Have you tried spiking your coffee? ; )
Ahhhh... you brought back memories with that comment.

Paul P: Thank you so much, but you've overestimated my abilities... I don't have many good ideas, but I work hard looking for the good ideas of others and then I try to play around with ways to communicate them. I really REALLY appreciate hearing that some of these are helpful to others -- it's why I enjoy it so much.

Cyndi: I suspected it applied elsewhere... I can't even imagine what stories you have about that ; ) [I could almost *see* your eyes rolling]

Posted by: Kathy Sierra | Dec 28, 2006 10:19:08 AM

This applies to hardware engineering as well.
We would make some Rev 1 prototypes out of big cardboard boxes.
Rev 2 would have medium sized cheap plastic boxes from Radio Shack.
Rev 3 would often be samples of something really close to the final product.

That seemed to show progress and set expectations that seemed to work some of the time.

Nothing like a 14 hour meeting spend discussing that 10x10x2 is "too big" for our product when the second prototype is only 5x4x2. The weird part is how some people latch onto something completely irrelevant and won't let go. 10x10x2 was the space under the passenger seat of a coworkers car and had nothing to do with the product but somehow the PHBs wouldn't stop talking about it for a 14 hour meeting.

Posted by: Stephan Fassmann | Dec 28, 2006 11:52:25 AM

You might like the discussion of the spectrum of specificity vs. abstractness and how that frames a user's perceptions and expectations in Scott McCloud's books: Understanding Comics and Making Comics.

Posted by: John D. Mitchell | Dec 28, 2006 12:05:39 PM

Alistair: Thanks for that link to the flow/state piece! I updated my post to include that at the end.

John Mitchell: That's a great reference -- the McLoud books. For anyone out there who hasn't read them, I'd suggest "Understanding Comics" as the most relevant for those who aren't actually in the comics business. I found "Making Comics" fascinating, but a bit more skewed toward the business of comics than the underlying principles which--as John gave an example of--relate quite well to what we do in writing, interaction design, product design, etc.

Thanks guys.

Posted by: Kathy Sierra | Dec 28, 2006 12:18:34 PM

Great post. Getting the prototype right is a critical task and is often more an art than a science. As is mentioned in your article, and most of the comments, the perspective of the stakeholder(s) is very different from the project manager, designers and developers.

My philosophy for the prototype is to nail the look-and-feel, branding, feature access and workflow. But the trick is to present it in a way that lets your sponsors know it's not finished, is not distracting, but does turn on the light for how the application will work.

One tip that works if you have a review coming up with a number of sponsors, is to identify one who will be able to be your guinea pig and make a test run. They should be able to tell you what was confusing, distracting, impressive and a yawner. You should be able to get tips on how to tweak it for certain other stakeholders - what's important to Sue isn't important to Bob.

Posted by: Ben Craigo | Dec 28, 2006 1:00:12 PM

Great post!

Ah...it reminds me of all the times I make a film for a client. When to show the client the film? Bare bones or finished?

It's always hard to tell ahead of time. I just wait for the process to unfold naturally and the answers always come at the right time.

Thomas R. Clifford

Posted by: Thomas R. Clifford | Dec 28, 2006 3:25:11 PM

Great post... one thing I struggle with is when do you show the stakeholders the actual pixel perfect version of what it is you're trying to build? My impression after reading this article is you wouldn't until the thing was actually built (thus reflecting reality).

In many large organizations however, execs and other stakeholders want to see high fidelty mock-ups of the UI before ordaining the rest of team to get started building it.

One approach I've recently tried is getting exec buy-off on page layout/navigation with rough wireframes, a high-level site map, and a style guide (provided by a grahpic designer). That way, I can seperate aesthetic feedback from architectural. It's too soon for me to say if this tactic works, so I'm curious to hear what others have done in similar shoes.

Posted by: ario | Dec 28, 2006 3:51:10 PM

Wouldn't it be logical to assume that if there was a ton of detail in the demo, that the creator had clearly settled on his foundation and would be *resistant* to any "big picture" type criticisms, and that part of the critic's motivation for focusing on smaller details would stem from the reluctance to stir up a big-bang scale conflict?

Posted by: Keith Handy | Dec 28, 2006 5:00:20 PM

That reminds me of a wonderful story in one of Alexander King's books. It seems as a young commercial artist, King was given an assignment to draw a party scene with about a dozen or so elegantly-dressed couples dancing. When he finished it, his mentor took one look at it and said, "Before you show it to the boss, put a big hairy man's hand on one of the women."

King refused. A day or so later, his mentor came back and saw him working on the drawing again. He said, "What happened?" King replied that the boss had told him he had to turn all dozen heads further towards the front. And the mentor said, "Yes ... if you had put the man's hand on one of the women, he would have taken one look at it and said, 'Oh my God! That's a MAN'S hand! FIX IT!'"

Posted by: M. Edward (Ed) Borasky | Dec 28, 2006 7:03:47 PM

ario: "That way, I can seperate aesthetic feedback from architectural..."
I know it's easier said than done, but yes... this is really ideal for most projects. I'm also interested in how others are doing, and I hope you let us know how your approach is working in the future.

Keith: Yes, absolutely right -- I think that's a really big part of the point (but I didn't make it clear)... that the exepectation of the reviewer is different when they think it's at a much later stage than it is. Regardless, I think it's always crucial to let reviewers/testers know the level of feedback you're looking for. And anyone who's ever worked in a waterfall-model project knows that requesting anything but small detailed changes at that point is probably out of the question.

I reckon, then, we have two big reasons why showing more "doneness" reduces the scope of the feedback: because the reviewer quite reasonably assumes it's not appropriate to do more, and because they're often not able to step back... it really depends on the domain and the expertise of the reviewer. With customer tests, for example, it's the same old problem--end-users may not know what they want until you actually show it to them, whereas a peer reviewer can look at it from a broader perspective.
Thanks for bringing that up.

Ed: That's a GREAT story! : ) And yes, it fits perfectly.

Posted by: Kathy Sierra | Dec 28, 2006 7:11:32 PM

Good post. I'd add a couple of other points:

- some significant % of "requirements" will never come up until users have a working product. These requirements are emergent - they're real but unknowable. No development technique will make this go away.
- There's probably a sweet spot, right after use cases but before perfect mockup, where you'll get most of your best information. Users often need to see something to begin to move into the imaginary world you're asking them to report on.

Posted by: Gary Furash | Dec 29, 2006 7:25:19 AM

The comments to this entry are closed.