Council of Canadian Academies reports on assessment of research performance

The Council of Canadian Academies (@scienceadvice) reported back last week on a year-long Expert Panel assessment of how research performance is measured in science and engineering disciplines. The 16-member panel included Sir Keith O’Nions (Rector, Imperial College London), and was co-reviewed by Claire Donovan (Reader, Brunel University). The specific question the panel was asked to report on was:

“What do the scientific evidence and the approaches used by other funding agencies globally have to offer, in terms of performance indicators and related best practices in the context of research in the natural sciences and engineering, carried out at universities, colleges, and polytechnics?”

You can read the full report on the CCA website. The Expert Panel’s main findings were:

• Many quantitative metrics (such as bibliometric indicators, research funding, researcher population and scientific infrastructure) can be used to assess performance at a national level.
• When used to assess funding allocations, quantitative indicators should inform, rather than replace, expert judgement.
• There is no “one size fits all” approach to research assessment, and therefore examples of international best practice are of limited use.
• Funding allocation is a complex process, and cannot be based simply on quantitative indicators.

These recommendations are similar to those in the UK’s renowned Research Assessment Exercise (RAE), which will be updated into the Research Excellence Framework (REF) in 2014, and the UK is used as a case study in the report. In the RAE, quantitative indicators are taken into account when assessing an institution’s research performance but ultimate authority lies with a panel of expert peer reviewers. The report notes that the RAE is generally acknowledged to have improved research quality and helped to develop the UK’s world-leading research base.

SIN Canada created an infographic on the 2008 RAE exercise, and are always happy to chat about science policy and excellence in UK research.