Noise vs. Bias

They say that if you ask a bunch of farmers to estimate the weight of a bull, the average estimate will come pretty close to the measured weight of the bull.

But, scientists say, you should not let the farmers know about the other guys' estimates.

That's what comes out of a very useful review of Noise: A Flaw in Human Judgment by Kahneman, Sibony, and Sunstein. It's reviewed at Quillette by Rob Henderson.

You need to trick people into making individual judgments, because if people know what other people think then they will tend to agree with them. Thus juries, panels evaluating employment applications.

In other words, if you are the second person evaluating a college application and you get to read the previous evaluator's comments, you will tend to agree with him. So shouldn't a university mask the comments of early evaluators?

The school’s reply: “We used to do that, but it resulted in so many disagreements that we switched to the current system.”

In other words "many organizations consider conflict avoidance to be at least as important as optimal decision-making."

So, if you are looking for "the wisdom of crowds" you need to make sure that crowd members don't know what the other crowd members are thinking. If you want "the madness of crowds" then you need everyone to know what the other guys are thinking. And chances are they will all agree with each other.

In politics, of course, 

The book reports findings indicating that when a group of Democrats saw that a particular view was gaining initial popularity among Democrats, they too would endorse that point of view, ultimately leading most Democrats in the group to support it. However, if they saw that a specific opinion was gaining popularity among Republicans, they rejected it. Republicans behaved similarly.

Not surprising.

There is also the time-of-day factor. Usually, it is best to get a decision from a decision maker early in the day, probably after their first cup of coffee. Later in the day they might not be so thoughtful.

Still, the same person can give a whole range of opinions on the same facts. They have done this with physicians. They are more likely to prescribe opioids at the end of a long day: the quick fix.  But studies show that "simple mechanical rules were generally superior to human judgment."

People tend to be forgiving when a human makes a mistake, and not forgiving when a machine makes a mistake.

If a human driver causes a traffic collision, we are often forgiving, depending on the circumstances. But if an autonomous vehicle causes an accident, people respond with intense suspicion. 

We expect machines to be perfect. Humans not so much.