Excellence vs. Relevance

Impetus for this blog post. Twitter h/t to Alex Lancaster (@LancasterSci).

In academia, the term excellence is often used in the context of scarcity and competitive dynamics (e.g. publications, career promotion), and as a result can be used quite arbitrarily [1]. In [1], a distinction is made between excellence and soundness. Excellence is seen as a subjective concept, while soundness (enabled through completeness, thoroughness, and an emphasis on reproducibility) is the adherence to clearly defined and practiced research standards. While it may also be true that the concept of soundness can suffer from the same subjective limitations, it is probably an improvement over the current discussions surrounding excellence.

Another term we rarely refer to, but may be of even greater importance, is the relevance of scientific research. In a previous post, I brought relevance theory to bear on potential biases in scientific selectivity. One way to think of relevance is as the collective attentional focus of a given research group, community, or field. Collective attention (and thus relevance) can change with time: papers, methods, and influences rise and fall as research ideas are executed and new information is encountered [2]. As such, relevance defines the scope of scientific research that defines a particular field or community of researchers. Given a particular focus, what is relevant defines what is excellent. In this case, we return to the biases inherent in excellence, but this time with a framework for understanding what it means in a given context.

There is also an interesting relationship between soundness and relevance. For example, the stated goal of venues like PLoS One and Scientific Reports is to evaluate manuscripts based on methodological soundness rather than merely on field-specific relevance. To some extent this has eliminated issues of arbitrary selectivity, yet reviewers and editors from various fields may still surruptitiously impose their own field-specific conventions to the review process. Interestingly, soundness itself can be a matter of relevance, as the use of specific methodologies and modes of investigation can be highly field-specific.

Sometimes relevance is a matching problem between an individual researcher and the conventions of a specific field. Relevance can be represented as a formalized conceptual problem using skillset geometries [3]. In the example below, I have shown how the relevance of a specific individual overlaps with what is considered relevant in a specific field. In this case, the researcher has expertise in multiple areas of expertise, while the field is deeply rooted in a single domain of knowledge. The area of overlap, or Area of Mutual Relevance, describes the degree of shared relevance between individual and community (sometimes called "fit").

How relevant is a single person's skillset in the context of a research community, and how do we leverage this expertise in an inclusive manner? The mutual relevance criterion might provide opportunities in cases where there seems to be a "lack of fit". Understanding the role of collective attention within research communities might allow us to consider how this affects both the flow of new ideas between fields and the successful practice of interdisciplinarity.


NOTES:
[1] Moore, S., Neylon, C., Eve, M.P., O'Donnell, D.P., and Pattinson, D. (2016). Excellence R Us: university research and the fetishisation of excellence. Palgrave Communications, 3, 16105. doi:10.1057/palcomms.2016.105.

[2] Wu, F. and Huberman, B.A. (2007). Novelty and collective attention. PNAS, 104(45), 17599-17601. doi: 10.1073/pnas.0704916104.

[3] First introduced in: Alicea, B. (2017). A peripheral Darwin Day post, but centrality in his collaboration graph. Synthetic Daisies blog, February 16.