The Double-Blind Review Is What Makes Decisions Fair

How does your team review ideas, project proposals, or paper drafts? Imagine I’m on your team and I need a budget allocated to a new project. I craft a proposal, elucidate the motivation, assess the risks, develop a plan, and then what? Do I create a PowerPoint presentation, present to my teammates for half an hour, answer their questions for another half hour, hear their honest feedback, after which they vote and a decision is made? If this is how things are organized within your team, you risk stifling creativity and motivation.

There are several problems with this fairly typical decision-making process in software teams or companies (with the most critical ones listed last):

Overload. Regardless of how crucial your proposal is, many meeting attendees have their own priorities. You cannot expect them to concentrate fully on your idea or actively engage in the discussion. Furthermore, some might just be lazy by nature.
Incompetence. No matter how talented your teammates are, expecting them to grasp every detail of your proposal in just thirty minutes and then evaluate it both objectively and professionally is unrealistic. It’s more likely they’ll base their judgments on personal feelings towards you.
Friendship. The decision-makers at the meeting are your colleagues. It’s challenging for them to provide unbiased and candid feedback since they don’t want to risk upsetting a friend. Very few can maintain both honesty and friendship—it’s a rare quality.
Jealousy. Since you’re presenting a proposal, it inherently becomes a contest for resources. You’re seeking either a project budget or a reallocation of team efforts. This puts you in opposition to others at the meeting. Many attendees might resent your initiative, even subconsciously attempting to thwart you.
Fear. If you’re the one presenting today, someone else might be in your shoes tomorrow. The same individuals critiquing your idea today might face similar scrutiny tomorrow. As much as we aim for professionalism, the urge for revenge can be strong, making candid feedback risky.

Considering these factors, it’s logical to presume that at the end of such meetings—especially if the decision goes against you—you might feel frustrated. You might view your team as lacking competence and feel that your talents go unrecognized. Best-case scenario: you decide not to present any new proposals. Worst-case: you begin seeking out a new team.

The issue isn’t necessarily with the team (they all are more or less the same) or your proposal (we all make mistakes). The crux of the problem lies in the decision-making process itself. Traditional meetings might not be the best avenue. Instead, I propose the following approach:

There is a review board of 10+ members and a secretary;
Proposals are submitted directly to the secretary;
The secretary randomly selects three board reviewers;
Each reviewer evaluates the proposal in writing;
Reviewers aren’t aware of the author’s identity;
They then submit their written feedback to the secretary;
A decision is made if 2+ reviewers endorse the proposal;
The secretary informs you of the verdict;
You don’t know the names of reviewers.

This method introduces two vital elements that can enhance the objectivity of decisions and increase proposal authors’ satisfaction: anonymity and authorship. On one hand, reviewers can offer genuine feedback without fear of backlash, knowing their identity remains confidential. Conversely, since their feedback is documented, they’re incentivized to remain professional and objective. Not knowing the author ensures decisions aren’t influenced by personal relationships.

Moreover, this system can elevate the quality of proposals. As authors can’t verbally elaborate during meetings, they’re compelled to articulate their ideas comprehensively in writing. The standard of feedback might also improve, particularly if the secretary mandates specific criteria for review summaries, as opposed to vague remarks.

A very similar review mechanism is practiced by scientific conferences. When you submit a paper to one, they don’t organize a meeting where you present it in an attempt to persuade them to accept your work. Instead, they ask you to double-blind your paper and then they send it to a few, more or less, randomly selected reviewers. When your paper is rejected (or accepted), you receive a detailed summary of the reasons, along with suggestions on how to improve it. If the conference is of a high level and its reviewers are doing their job correctly, you will be satisfied even in the case of rejection.

In conclusion, the suggested process can lead to superior decision quality and boost the motivation of those proposing ideas. Even if some suggestions are declined, they’re more likely to return to the review board with renewed proposals.