Contribution to the independent review of the Outcome Based Investment Process

A policy document by the Royal Society of New Zealand Biodiversity Committee

INTRODUCTION

Herein, we respond to the three questions raised by OBI reviewer David Webber and also give some general comments based on feedback from committee members on the Outcome Based Investment process.

The Royal Society of New Zealand (RSNZ) Biodiversity Committee comprises 11 scientists, representing 3 Crown Research Institutes (HortResearch, Landcare Research, NIWA), 2 government departments/ministries (DoC, MAF Biosecurity Authority), 2 universities (Auckland, Massey), the Botanical Services Curator of Christchurch City Council, a retired phycologist (formerly with Landcare Research), and a RSNZ representative. Not everyone was involved in the OBI process and, therefore, not everyone personally contributed to this response, although each has had the opportunity to provide feedback, and, so, may be considered the consensus view of the Committee. We have, in addition, consulted colleagues who were involved in the OBI, and some of their thoughts and experiences are reflected in our response.

The RSNZ Biodiversity Committee was established in 1994 as a result of the perceived important role for science concerned with biological diversity, particularly in the light of obligations consequent to the signing of the UN Convention on Biological Diversity. Our Terms of Reference are:

Having regard to the Convention on Biological Diversity, the full range of land tenure and use in New Zealand, and in both terrestrial and aquatic ecosystems, the committee shall:

1. Identify scientific, technical and educational programmes contributing to biodiversity research with special reference to:
   • Identification — systematics, taxonomy, and paleoecology
   • Conservation — monitoring systems, ecosystems analysis
   • Sustainable use — economic and social impacts
2. Establish priorities for research in biodiversity having due regard to the capabilities of research providers, future needs and the requirements of users.
   • Facilitate application of biodiversity research in the SW Pacific and Antarctica
   • Contribute to preparation of a biodiversity strategy for New Zealand [done]

In this document,

• OBI = Outcome Based Investment;
• FRST = Foundation for Research, Science and Technology, the government agency administering the OBI;
• MoRST = The Ministry of Research, Science and Technology

Our response

In general terms, we applaud the overall goals “to enhance and support the resilience, functioning, and recovery of land, freshwater, and marine ecosystems”, not that there is anything new in this – these were implicit goals of the former DSIR and have been explicitly stated since. We also endorse the six target outcomes for the ecosystems area:

• Define New Zealand’s biota
• Reverse the decline in New Zealand’s indigenous biodiversity
• Biosecurity—management of incursions
• Biosecurity—management of existing pests
• Protection of unique ecosystems of Southern Ocean and Antarctica
• Sustainable use of aquatic and terrestrial biota

As science professionals, we value opportunities to do quality science in the public good. Our goal as environmental scientists has always been to understand the components of New Zealand’s ecosystems and the dynamics of ecosystem processes, to influence human behaviour in relation to ecosystems, and assess risks to environmental well-being. Accordingly, we support the goal of dialogue with relevant end-users (again, nothing new in this common practice for former DSIR scientists) and engaging them in the application of scientific knowledge to environmental goals. The OBI process should be aimed at helping scientists, policy-makers, and end-users achieve all this.

1. The design of the OBI model as a funding mechanism

The overall concept of OBI is positive in seeking to move very deliberately beyond science outputs to implementation and achievement of intermediate outcomes. As recognised in the 2004 PCE report (Missing Links: Connecting science with environmental policy, [1]), there has been a disconnect between scientists and policy-makers in recent years. Currently, aside from a few largely ineffectual “advisory groups”, the primary relationship between researchers and policy/management agencies is primarily based on contract negotiations and provision of associated reports. It is important for MORST and FRST to engage more closely with science providers, especially in regard to developing longer-term scientific strategies for achieving desired environmental outcomes. Effecting fruitful links with users (other scientists plus end-user groups) should proceed from this foundation.

In 2004, MoRST evaluated New Zealand’s environmental RS&T system up to 30 June 2003[2] to determine if the Government is getting good value from its investment in environmental RS&T. The evaluation found that environmental RS&T underpins a lot of economic activity, especially, inter alia, activities that use or impact on fresh water and marine activities, are important for natural-hazard management, and inform important government work programmes such as the Biodiversity Strategy. It was asserted that links between researchers and end-users could be improved, viz “a majority of end-users could gain greater benefit if they were better connected with the RS&T system”.

The shift to OBIs aims to result in more and stronger connections among research providers and policy/management agencies through a shared focus on achievement of agreed and auditable outcomes. Yet FRST by itself does not have the power to make improvements outside the science system and risks penalising science providers if end-users fail or choose not to effect outcomes through their use of scientific knowledge. It should be noted, too, that obligating science providers and users to be in very close contact will not always result in the best outcomes; there is a need for science to be able to maintain some independence (e.g. from pressures of competing interests such as conservation versus the fishing industry).

There seems to be a strong but misinformed perception that there are not good connections between end-users and environmental research providers (e.g. MoRST 2004). Yet, a review of technological learning (FRST 1998, [3]) suggests that good connections do exist between many environmental end-users and research providers. Where connections and awareness are not good, there is a failure to be clear about where the problems lie, as it often does not lie with the scientists but with the capacity of end-users to access and make use of scientific information. Given the internet and a multiplicity of commercial abstracting services and online specialist databases, any end-user criticism about inadequate communication and information seems hollow. MoRST (2004) has noted that many end-users do not have sufficient scientific capability to use the science as it is currently presented. Is fixing that weakness the best use of the skills of science professionals?

Beyond the goal of connectedness, however, we question whether the OBI funding model will be effective at enhancing scientific excellence and capacity in New Zealand – there is a disconnect between aspirations and reality. We caution that, unless appropriately managed, the OBI funding model is no guarantee of enhancement of scientific excellence and capacity in NZ. We note that scientific excellence as a funding criterion appears to be rated relatively lowly in the “due-diligence” assessment and suggest that this could be interpreted as a decrease in emphasis on the critical role science excellence plays in improving environmental management performance. Further, if the OBI mechanism leads to narrowly prescribed contractual-style work to address operational needs of a limited set of “end-user agencies such as local, regional, or central government organisations” science excellence will be an early casualty. To mitigate against such an outcome, the role of strategic science direction and the importance of maintaining systems to monitor and manage science excellence within OBIs must be made a more explicit component of OBI evaluation. While these systems can be used to manage science excellence on a day-to-day basis, regular independent evaluation of the overall OBI science strategy and performance should be a key component of the 4-year OBI review cycle. A case in point – biocomplexity is one of the most challenging areas of ecological science, policy and management today. Our lack of understanding concerning the function and resilience of NZ’s ecosystems precludes specification of an explicit policy and management context for the research that we might undertake in this area. Under these conditions, more emphasis needs to be placed on the scientific skills, methodologies, hypotheses to be tested, and international collaborations, than on the operational or policy needs of agencies charged with managing ecosystems. While in the first instance the immediate end-users are other scientists, in due course, this sort of research will yield further local and national benefits. However, the challenge for OBIs in this area is firstly to ensure that the requisite science gets done.

Among the key findings of MoRST’s evaluation of environmental research were that the demand for environmental RS&T from both government (local and central) and industry sectors is likely to increase to support sustainable economic growth. However, funding for many areas of environmental research has remained static since 1998 and in some areas since 1993 (i.e. declined in real terms). (Taxonomy is one example, but there is no national biosystematics strategy to mentor the next generation of taxonomists and ensure continuity of skills.) Overall, the number of FRST-funded scientists in NIWA and Landcare Research (the two primary environmental research CRIs) has declined by 20% since 1998. As a result, some underlying core capabilities (and potential scientific productivity owing to more time spent on commercial work) have declined. The 2005 budget made no more money available for environmental research, in real terms, considering inflation. While it has been noted that commercial revenues for NIWA and Landcare Research have increased since 1998, suggesting that increased use is indeed being made of existing knowledge; these increases are associated with commercial activity and investments in underlying core science capabilities still need to be increased. Can the OBI funding process guarantee this?

The MoRST report also noted the absence of clear directions for environmental research. The New Zealand Biodiversity Strategy (NZBS) specifically brings attention to the lack of any aligned science strategy, to ensure achieving the NZBS 2020 goals. Policy research capability in New Zealand is limited. This limitation may impact on the implementation of some environmental programmes. New Zealand must arrive at an appropriate match of policy, long-term research strategy, and funding. The long-term effectiveness of any OBI funding process must be seen in this larger context.

We are sceptical that the OBI method of governance will achieve whole-of-government objectives any better than they were being achieved in the past. Instead, we foresee a fragmentation of research effort (that will not support strategic long-term programmes), governance marred by infighting between stakeholders with competing interests and objectives, incentives to finance operational applications of research from the RS&T system, a growing administrative burden on already cash-strapped research, growing job dissatisfaction among scientists, and a negative impact on the quality and quantity of good science that is done.

2. The implementation process as applied to the ecosystems round over these last few months

Some strongly worded responses were received in relation to this question, especially from a member of the Expert Advisory Group that was consulted about the Ecosystem Portfolio prior to the release of the Request for Proposals. The issues are as follows:

1. The process was perceived as being unnecessarily costly in time and resources at all levels. The members of the Expert Panel believed they were involved in a process where they were being seriously consulted — but almost none of the messages emphasised by the Group have been reflected in the process or outcomes to date. The inordinate amount of time spent on the process, which if successful, will, ironically, result in less time and fewer resources available for some of the very topics that FRST cited as being important in the Request for Proposals (e.g. taxonomy).
2. The time loss also extended to the way in which FRST raised expectations of people previously in the pool and newcomers to the pool; they were actively recruiting for new applicants when the available money was shrinking. (We understand that it was 2.7 times 'overbid'.) This led to stresses and tensions between groups that had been collaborative and who were forced into competitive modes. Several respondents noted an enhanced level of competition and stress, even among individuals in different organisations who had previously collaborated well. Time loss also resulted from FRST’s slow release of documents informing applicants of what was required. It appeared that FRST was developing new processes on the hop, evolving the structures and requirements even as people were trying to apply to the portfolio. The lack of clarity and the mixed and differing messages coming from FRST was also a serious short-coming of the process.
3. The use of a portal for the final application stage added many hours of additional time. It seemed really pointless – perhaps a case of an inappropriate technology being inflicted on the applicants. It highlighted that its originators did not understand the process from the applicants’ end.

A concern arising from the OBI process as it was experienced is the likely cost of the bureaucracy involved in funding public-good science. Is there an accountability process? By stacking a very significant bureaucracy onto the science providers, one suspects that the resources available for science are decreased even further. Given the small population size of New Zealand (less than many overseas cities), the extraordinary bureaucracy involved in regulating science funding begins to look like the proverbial sledgehammer to crack a nut.

3. The applicability of the model to other areas of government funding of research in the future

Unless there is significant improvement in 1) developing long-term perspectives on science need, science training, and science capacity and 2) the bureaucratic process, we do not recommend that the model as it currently exists, be applied to other areas of government funding of research.

The conclusion in the Evaluation of the Environmental Output Class that “there is an absence of clear research directions for environmental research” (MoRST 2004) is germane to the OBI process. But it has become unclear which organisation has the primary role in leading the development of research directions and strategies and in facilitating the input of all stakeholders (including scientists) in environmental science (and other areas). Environmental scientists would be greatly reassured if there had been more evidence of strong links and concordance between, for example, PCE, MoRST and FRST when the OBI pilot was implemented.

The RSNZ Biodiversity Committee strongly approves of MoRST’s (2004) recommendations, specifically endorsing those reproduced below. These should inform the OBI funding process.

Recommendation 1: That MoRST, in conjunction with other key stakeholders such as the central government agencies that have accountabilities in the natural resources area, and local government, provides improved direction for the science system by developing an environmental research priorities statement. [Inter alia, we suggest a science plan aligned with the 2020 goals of the New Zealand Biodiversity Strategy.]

Recommendation 5: That MoRST clarifies the roles and responsibilities of the main actors involved in environmental RS&T.

Recommendation 6: That MoRST investigates advantages of, and options for, increasing the differentiation of environmental research funding processes along the lines of long-term, applied, tools; and policy research as part of work to clarify roles and responsibilities.

Recommendation 7: That MoRST works with FRST to review FRST’s environmental communications strategy, with the aim of developing much clearer communications with the environmental science system.

---

1Missing Links: Connecting science with environmental policy. New Zealand Parliamentary Commissioner for the Environment, September 2004. [ISBN 1-877274-52-6.]

2Evaluation of the Environmental Output Class. New Zealand Ministry of Research Science and Technology, December 2004.[pdf]

3 Technological Learning and Knowledge Application Review. Foundation for Research, Science & Technology, 1998. [link]