I like the using the notion of entropy to describe a wide range of peer-review opinions based on the same piece of work. This reminds me of the "bifurcating opinion" phenomenon I sketched out a few years ago [1]. In that case, I conceptually demonstrated how a divergence of opinion can prevent consensus decision-making and lead to editorial deliberation. Whether this leads to subjective intervention by the editor is unclear and could be addressed with data.
Hiroki points out that "high-entropy" reviews (wider range of opinions) represent a high degree of innovation. This is an interesting interpretation, one which leads to another Twitter conversation-turned complementary blog posts from Michael Neilsen [2] and Julia Galef [3] on the relationship between creativity and innovation.
In my interpretation of the conversation, Michael point out that there is a tension between creativity and rational thinking. On one side (creativity) we have seemingly crazy and irrational ideas, while on the other side we have optimal ideas given the current body of knowledge. In particular, Michael argues that the practice of "fooling oneself" (or being overly confident of the novel interpretation) is critical for nurturing innovative ideas. An overconfidence in conventional knowledge and typical approaches both work to stifle innovation, even in cases where the innovation is clearly superior.Hiroki points out that "high-entropy" reviews (wider range of opinions) represent a high degree of innovation. This is an interesting interpretation, one which leads to another Twitter conversation-turned complementary blog posts from Michael Neilsen [2] and Julia Galef [3] on the relationship between creativity and innovation.
Feynman though that "fooling oneself" was generally to be avoided, but also serves as a hallmark of scientific rationality. However, the very act of thinking (cognitive processes such as focusing attention) might be based on fooling ourselves [4], and thus might define any well-argued position.
Julia disagrees with this premise, and thinks there is no tension between rationality and innovative ideas. Rather, there is a difference between confidence that an idea can be turned into an artifact and confidence that it will be practical. Innovation is stifled by a combination of overconfidence in practical failure combined with a lack of thinking in terms of expected value. I take this to be similar to normative risk-aversion by the wider community. If individual innovators are confident in their own ideas, despite the sanctions imposed by negative social feedback, they are more likely to pursue them.
Nikola Tesla's approach was "irrational", it was also a sign of his purposeful self-delusion and perhaps even his social isolation from the scientific community [5]. Remember, in the context of this blogpost, these are all good things.
Putting this in the context of peer review, it could be said that confidence or overconfidence is related to the existence and temporary suspension of sociocultural mores in a given intellectual community. A standard definition of social mores are customs and practices enforced through social pressure. In the example given by Michael Neilsen, fooling oneself in order to advance a controversial position requires an individual to temporarily suspend social mores held by members of a specific intellectual community. In this case, mores are defined as commonly-held knowledge and expected outcomes, but can also include idiosyncratic practices and intuitions [6]. From a cognitive standpoint, this may be similar to the requisite temporary suspension of disbelief during enjoyable experiences.
While this suspension allows for innovation, violations of social mores can also lead to a generally negative response, including moral panics and the occasional face full of bees [7]. Therefore, I would amend Hiroki's observation by saying that innovation is marked not only by a wide range of peer-review opinion, but also by universal rejection. Separating the wheat from the chaff amongst the universally rejected works is work for another time.
The price of innovation equals a swarm of angry bees!
NOTES:
[1] Alicea, B. (2013). Fireside Science: The Consensus-Novelty Dampening. Synthetic Daisies blog, October 22.
[2] Nielsen, M. (2017). Is there is tension between creativity and accuracy? April 8.
[3] Galef, J. (2017). Does irrationality fuel innovation? Julia Galef blog, April 7.
[4] Scientific American (2010). How We Fool Ourselves Over and Over. 60-second Mind podcast, June 19.
[5] Bradnam, K. (2014). The Tesla index: a measure of social isolation for scientists. ACGT blog, July 31.
[6] Lucey, B. (2015). A dozen ways to get your academic paper rejected. Brian M. Lucey blog, September 9.
[2] Nielsen, M. (2017). Is there is tension between creativity and accuracy? April 8.
[3] Galef, J. (2017). Does irrationality fuel innovation? Julia Galef blog, April 7.
[4] Scientific American (2010). How We Fool Ourselves Over and Over. 60-second Mind podcast, June 19.
[5] Bradnam, K. (2014). The Tesla index: a measure of social isolation for scientists. ACGT blog, July 31.
[6] Lucey, B. (2015). A dozen ways to get your academic paper rejected. Brian M. Lucey blog, September 9.
[7] "Face full of bees" is a term I just coined to describe the universal rejection of a particularly innovative piece of work. "Many bees on face" = "Stinging rebuke".
