Scientific Assessments: matching the approach to the problem

Scientific Assessments: matching the approach to the problem
Robert J. Scholes

Scientific Assessments are a large part of the lives of many researchers. They aim to be an interface between science and society; and particularly between science and policy. As the nature of the problems demanding policy action becomes more and more technical and complex, so the need has arisen to have a process to collate and evaluate the relevant information, and then communicate it to policymakers, along with the associated uncertainties, in a clear fashion. There is a range of ways to conduct such assessments, which vary in how long they take, how many (and what type) of assessors they use, and how they engage with the users. The paper outlines some of the key attributes and success factors of several assessment approaches, and matches them to the circumstances where they are most likely to be successful.

Where the audience is other researchers, and the question being addressed is an emerging area in science, the scientific review is a well-established approach, taught at universities, and following certain conventions. It is performed by one or a few researchers, who cover a usually narrow area quite exhaustively. They tend to leave the assessment of what is reviewed to the reader. The approach is appropriate for an issue which is of scientific interest, but no great social prominence.

Where the science is relatively straightforward and well-established, and a quick but well-focused input to policy is needed, the ‘Briefing Note’ is a well-tested vehicle. It is typically written by one person, but benefits from both peer review, and an initial interactive process to define the scope. Brevity, focus and clarity are of the essence.

More complex issues, where the science may be difficult but is not in dispute are typically handled with a Contract Report: the team of experts is multidisciplinary but relatively small, and a single peer-review step is used.  Often new data is presented. A summarizing overview is helpful.

Where the public interest is high (and often polarized), the issue is complex (involving many sub-questions and disciplines, interactions and conditional outcomes) and the science is quite technical and often incompletely settled, the appropriate approach involves much greater two-way engagement with user groups for the assessors to understand policymaker’s needs, and for the policymakers to understand the strengths and limits of the science, and to participate in the selection of the expert teams. The opinion of the experts is actively but transparently sought; a multi-author team (often numbering hundreds) helps to provide a balanced assessment, along with extensive (repeated), documented and open review. A solid authorizing environment is critical for success. 

Read 735 times Last modified on Tuesday, 03 May 2016 12:09