Nature 460, 949 (20 August 2009) | doi:10.1038/460949a; Published online 19 August 2009
http://www.nature.com/nature/journal/v460/n7258/full/460949a.html

Conor O’Carroll1

Irish Universities Association, 48 Merrion Square, Dublin 2, Ireland
Email: conor.ocarroll@iua.ie

Sir
Your News story ‘Italy outsources peer review to NIH’ (Nature 459, 900; 2009)
highlights a problem common to many countries with a small population of
research scientists. Ireland can be held up as a successful model in addressing
this problem because, over the past eight years, funding agencies have moved to
fully international peer review.

A few years ago, important research and development ventures were set up with a
new infrastructure to attract talented people from abroad. The use of only
Irish peer reviewers to allocate millions of research euros to a small number
of universities could not stand up to the principles of objectivity,
transparency and perceived fairness and would have led to conflicts of
interest. Despite initial opposition, exclusively international review is now
accepted; researchers want to be benchmarked internationally as well as
nationally.

The typical process for research evaluation in Ireland is to consult four or
five reviewers by mail for each proposal. Proposals are then assessed by a
panel of invited experts, who meet in Ireland. Reviewers may be sourced through
international funding agencies, or by letting applicants nominate experts
themselves.

Some Italian scientists in your News story express reservations. They may well
have a point, as US reviewers will probably not have any detailed knowledge of
how research is conducted in Italy. One approach is to have nationals involved,
either as observers or in a formal non-voting role. For example, the Irish
Health Research Board (http://www.hrb.ie) organizes international mail reviews
and panels, but the chair of each is Irish. They cannot participate in
selection, but ensure that the correct procedures are followed and can explain
the national research-funding policy. International panel members appreciate
this local input, which helps them think outside their own national funding
system.

Reviewing criteria often include the quality of the project, the researchers and
their institutions, and the social and economic impact of the research. It is
important that international reviewers focus on the quality of the first two,
as the standing of institutions and the probable impact of a project can be
harder for them to evaluate. Also, they should not get involved in detailed
budgetary considerations, as these are strictly national.

Things have changed radically in Ireland’s research over the past ten years. In
2008, the country appeared for the first time in a list of ‘Top countries in
all fields’ (ranked by citations per paper; http://tinyurl.com/m5wdcl). We are
now placed 19th, up from 36th place in 2003. I believe that international peer
review played a significant part in this development.