In my experience with software development at bigco, it's the lead architect-type person, who has the direct ear of upper management and has already made a decision well in advance of the developers/designers even hearing about the project. This decision will not be communicated in advance, rather people must generate several good ideas that will then be pared down to something close to what the architect originally had in mind, though they will all be moderately unsatisfactory due to the obvious limitations in this process. The result is a product that no one is completely happy with but at the same time no one can completely fault because of the extensive research and consensus that went into its development. This lack of enthusiasm will trickle down through the rest of the project until everyone is just looking for a way to get onto some other, newer project that is still untainted yet will inevitably undertake the same cycle in due course.
If a truly excellent idea should arise from the independent investigation, it will be shelved for a 2.0 product release due to the "perceived risk" or "out of scope requirements".
From the outside, however, this process looks exactly like the diagram in the article.
Pardon my cynicism, I suppose this is why I got fed up with commercial software development...I hear it's not like this everywhere;)
I love discussing implementation details of different designs, but it has always been to my disadvantage to acknowledge what's good about a competing vision and lay out in detail the pros and cons of my own idea, because that just gets turned around to support the high-level idea of the senior employee who's been with the company longer (and has already earned the trust of the IT director).
The devil is always in the details, so I love discussing all "gotchas" before code is written. But I guess early in the project, people like to believe that the new design will solve all problems and will only take two months to implement, and a frank discussion can only dampen that enthusiasm. So it tends to get suppressed rather than encouraged.
How could we avoid this? The article says it should be the one who is better at self criticism. But how do we unbiasedely pick this person? You imply non-commercial development is different. How so?
Let me give an example of LSMB's newer architecture which is still being refined.
One member says "Hey, we should use stored procedures and here's why." I say "No, we shouldn't and here's why." We argue back and forth. We end up using stored procedures in a way that avoids my objection. We both win.
One member says "Hey, we should ditch LaTeX for Docbook XML." I say "No we shouldn't because then I have to spend time I could be spending programming learning a new toolchain and documentation master format." He says "No worries. I am sure someone else can maintain the docs." I say "Great." But nobody does so we stick with LaTeX.
The goal outright has to be to ensure that everyone's legitimate objections are met to the extent possible....
A better approach would be for the architect to make the first proposal. "Here's what I am thinking. Any objections? Any criticism?" And then fire off a discussion as to what the shortcomings of the approach are. These can then be the subject of a second iteration or where the architect really is wrong he can go back to the drawing board. Ideally this is done before going to upper level management.
The only problem here is that opening up your ideas to such criticism takes a sense of security and confidence that is lacking in most management types in corporate America today. People's whose primary skills include playing politics don't tend to be confident standing on their own...
Now, here's an example. In LSMB we added the ability to store files as attachments to transactions. I sent off a proposal and a few people liked it and one person really didn't. He talked a bit about how to detect and reduce file duplication in the database. There were some low-level I/O problems with his proposals, but he had other substantive objections which were correct. So we went back and forth on this a while and the end result was that I came back a week later with a new proposal taking a lot of that into account.
Picking great leaders? That's a tough one. An easier question is to ask "What prompts a Lead 'Hat' stick with an idea though it doesn't seem to be the 'best' idea?"
Developers/designers using OP's two-way criticism style are really removing their own egos and emotions from the discussion and letting ideas honestly fight for themselves. I suspect that the Lead "Hat" is a conduit between developers/designers and executives and it's naturally difficult to convey, let alone objectively discuss corporate objectives in a development/design meeting.
The takeaway for Lead Hat lurkers on HN would be to (unfortunately) expose developers/designers to a wider scope of the project when the ideas are fighting, so they can understand where you're coming from.
Productive self-criticism is a hard thing to select for. It shouldn't be that hard to determine if someone has really thought through the ramifications of their own product design, but it can be much harder to determine if they've really thought through someone else's. Organizing people across teams any larger than a few people is a also hard problem, one for which I unfortunately don't have a ready solution. In small teams, this kind of thing is much easier to spot and, hopefully, correct.
One way would be to do some prototype and demo your idea.
At the same time it is no guarantee that this would work all the time. It depends on factors like how cohesive the team is, whether there is openness to listen to conflicting thoughts, ideas etc. Otherwise your prototype might get as well shot down with some specious argument.
I think one of the most revealing moments in my design education was when a teacher asked us to design a packaging that looked cheap. We all felt relief because we thought it meant we didn't have to be as careful with the quality of our design, and in general most proposals were shoddy and poorly built.
Of course, the teacher chided us for being careless, "looking cheap" doesn't mean low quality, it means subtle signals like inexpensive materials, no flourishes, no 'status symbol' indicators.
When looking at certain design pieces we often have a visceral reaction, you either like it or you don't. You must step back and think about the purpose, if the design accomplishes its purpose then it's good design. The aesthetics might be off, but that's easy to fix.
On that note, IKEA is the only company I can think of that successfully designs their products and branding to look "cheap", as you define it. Countless other companies seem to go for a more "trashy" look, akin to tabloids.
In Design school, we learned how to criticize without being jerks. We also learned to trust other people, and take criticism without being offended.
After 4 years in an environment like that, I'm amazed at the design process in the startup world. People cling to their pet projects -- and they either don't criticize for fear of hurting someone's feelings, or they take criticism very personally.
I take this skill set for granted, but there aren't that many places to learn how to give and take criticism, and many companies have not figured out how to embed it in their culture.
> In Design school, we learned how to criticize without being jerks. We also learned to trust other people, and take criticism without being offended.
I want to design school also. One thing I noticed right away:
We were _taught_ those things, but a lot of people didn't _learn_ them. Even near the end of the program, many of my classmates couldn't handle well (or usually just ignored) the feedback they got in group critiques.
Mostly, it was a problem that solved itself. The people who learned to give and take criticism improved a lot more than those that didn't.
The most important piece of advice is the last line of the article.
In my experience, ignoring the above combined with a refinement only approach generates some amazingly horrible 'solutions'.
An example of one such 'solution' I was forced to implement by a manager and 'senior developer'. To make matters worse I was expected to be the one to supervise/troubleshoot the use of this 'solution' every week. Here goes:
Output data on a unix server, ftp it to a Windows server, ftp to a Windows PC (yes, it was FTP'ed twice on the same LAN), manually change in Access and export, ftp back onto the server, process again, FTPx2 back to the Windows PC, email to an off-site processor, wait 2-5 hours for a response, ftp back to the Server, process, ftp back to Windows, put it into Access again and export, and then ftp it to the clients mainframe (and manually remember to add some archaic FTP settings). On average it took 1.5 hours of dedicated work time to complete, not including the wait time or reattempts. It was extremely error prone due to the number of manual steps. Changing the settings in ftp also caused problems with other clients when people forgot to take off the archaic mainframe ftp settings.
For my own sanity I assumed responsibility of the task being done each week and replaced it with a script that did everything in half a minute including sending it off for offsite processing - the results of which I never used. I was eventually found out when the client requested the data early and I provided it to them several hours before the offsite processing was done. I got in trouble because of the thank-you email.
If we're ever in a design meeting together, please, please don't act like this.
I already know the benefits of my design, that's why I presented it. I don't need you to point them out again. I need you to criticize my design and think hard about edge cases I've missed. Don't waste time trying to think of something nice to say, spend your time trying to tear my design apart.
I might feel bad to have the flaws pointed out, but I'll feel a hell of a lot worse if we go forward with my design only to find a devastating roadblock months down the road.
The thought process we go through to work out "what circumstances your design is good for" highlights problems. (e.g. "David, your navigation really well when there is a predictable number of categories, unlikely to change") highlights the shortcomings implicitly (e.g. "But, we're designing a wiki here, anyone can add anything.")
It sounds like you're a mature designer capable of taking criticism without any issue whatsoever. Most designers aren't like that, and unfortunately many who are can be difficult to work with because they think everyone handles criticism as "well" as they do.
I tend to agree. It can depend though on circumstances and personality types. I think we can adjust feedback methods depending on who is involved, how well we know each other, and so on.
That said, some people try too hard "not to be a dick", and while nice and polite, they end up supplying watered down, fence-sitting, egg-shell treading fluff instead of useful down-to-earth efficient feedback.
Some designers get emotional. That's good - but they need to learn to channel that emotion into reasoning and solid foundation arguments to support why they did what they did. If they can't do that, they need to work with people who can assist them in doing that.
A colleague once used "intuition" as her rationale for a design/IA decision that was going to negatively impact on technical work I had to do. My criticism was harsh in reply, and I'm sure she thought I was "a dick". Well, that's just tough. Build a bridge and let's move on to the next task. All intuitive decisions can be unscrambled or decoded to find their basis. Especially true in design environments where you're working within defined limitations, scope and objectives. If you can't find a basis, then unload the criticism, keep our humor, and let's solve the problem rather than mess around.
> by making a rule of praising alternate solutions and criticising your own, the discussions move clear of the realm of personal preference and bias. It's simply a discussion of what is right, and when.
How strict should this rule be? Where does it end, and where does legitimate criticism begin?
It is an illusion that people can simply talk about "what is right and when" without getting caught up in biases. Biases just get subtler and therefore more difficult to catch. Everyone who strongly favors one or another solution does so because they think it's the right solution, not merely because it's their favorite solution. You can't escape that by simply declaring that you're going to talk about what the right solution is. Meanwhile, people are often incapable of seeing all the shortcomings of their favorite idea, because it's their favorite idea. When that happens, you need other people to criticize it for you.
A rule that tells you to criticize your own idea and flaunt the benefits of other ideas could lead to suboptimal solutions when people try too hard to please one another and not all of the shortcomings are identified. What if you know that the other guy's solution will break down in an edge case that he seems to be totally oblivious of? The proposed rules of the discussion subtly discourages, if not prohibits, you from mentioning it until a later time when it might be too late. Of course you shouldn't be a hothead all the time. But if the only alternative is to express your discontent in a passive-aggressive, underhanded manner, how is that better than, for example, using Crocker's Rules [1] ?
But I guess it ultimately depends on the personalities of the people involved. If you've got a bunch of hotheads, maybe they really need to cool down and start thinking positively. But there are always some people who are really good at spotting problems but who are afraid to hurt anyone's feelings. You really don't want to discourage them any further.
I suspect the point is that "following the rule" will lead to introspection and self-criticism. Obviously rote adherence to any rule set isn't going to lead to good design by itself.
As a rule you should be capable of introspection and you should look at what works and when for other peoples design.
Regarding following it "strictly", depends on the circumstances. If a new designer joins the team, and doesn't do it, they should be let know. If someone's design is universally regarded as the absolute best and a masterpiece, sure skip the crit and move on.
This sounds a bit like the complement sandwich: start with a positive (where it works great), then criticize (where it falls short) and wrap up with another positive. Oddly enough, even though its a bit clichéd, it is a remarkably effective little tool.
This is great because it forces participants to carefully consider the _details_ of each others' designs.
As a designer moves along he or she will invariably encounter lots of little constraints that were previously unknown. A good design is a response to all of these details. As a reviewer its all too easy to dismiss a design: "you should have just done X". But the designer will respond "it was considered but I found extra requirements Y and Z that preclude X."
Communicating these details is difficult. You may not always remember all of the constraints you've discovered. A process like the one posted should help to get everyone thinking about the details discovered by each individual.
Hmm.. this reminds me of a design presentation I saw a while back that talked about using the "6 Thinking Hats" approach. Basically, after creating a number of designs, a facilitator walks team members through six different ways of thinking about a design. I guess it covers the same ideas as "walking both sides of the street", but it was suggested that 6 hats is easier to apply when dealing with non design personnel.
One surprising knock-on effect of this approach is a reduction in pointless design arguments. The ones that are rarely constructive, where people get offended and cling on to their own precious concepts.
I think the strategy described is elegant in its directness. It's just a simple inversion of people's natural tendencies -- the very ones that get people into pointless arguments.
> "The person who demonstrates most knowledge about the shortcomings of their own solution and the benefits of all the alternatives is the best best equipped to make the call."
I like this approach of having the person who points out all of the good in others' designs and the flaws in their own designs is best qualified and/or unbiased to make the call on the "best design."
How do you identify this person though? Isn't "the person who demonstrates most knowledge about the shortcomings of their own solution and the benefits of all the alternatives is the best" a subjective determination? What if a handful of people are all demonstrating this ability? Do you have all of them "make the call" by committee? That seems to defeat the purpose of this exercise...
Hey Luke,
I see where you're coming from. It is subjective, but the key benefit is really that people are more likely to naturally cede to the person most equipped to make the call. The argument is really about establishing that. If it comes down to a 1 v 1 situation where two guys are at loggerheads, then yeah, some design principal needs to be step in and make a call. But the process hopefully reduces how often this is necessary.
From personal experience those who give the most useless kind of criticism tend to be very aggressive when receiving even the mildest form of constructive criticism, or even suggestions.
In my experience with software development at bigco, it's the lead architect-type person, who has the direct ear of upper management and has already made a decision well in advance of the developers/designers even hearing about the project. This decision will not be communicated in advance, rather people must generate several good ideas that will then be pared down to something close to what the architect originally had in mind, though they will all be moderately unsatisfactory due to the obvious limitations in this process. The result is a product that no one is completely happy with but at the same time no one can completely fault because of the extensive research and consensus that went into its development. This lack of enthusiasm will trickle down through the rest of the project until everyone is just looking for a way to get onto some other, newer project that is still untainted yet will inevitably undertake the same cycle in due course.
If a truly excellent idea should arise from the independent investigation, it will be shelved for a 2.0 product release due to the "perceived risk" or "out of scope requirements".
From the outside, however, this process looks exactly like the diagram in the article.
Pardon my cynicism, I suppose this is why I got fed up with commercial software development...I hear it's not like this everywhere;)