April 19, 2011
This is the first of my three wishes for the FP8 foresight process. Wish 1: Stop demanding that EU-funded research be evaluated on impact.
Any FP7 research proposal is presently evaluated on at least three main themes – one of them being impact. This might seem like common sense in the project management world: any research funded with public money should make an impact on society, the quality of life and a better tomorrow. I don’t agree though that impact should have such a primary position in how the EU funds its research.
Problems with stressing impact include:
- Impact proposals imply that scientists can state clearly at an early stage what their research will produce. This is unrealistic – some research only pays off decades later and, in any case, the EU should be funding long-term research (short term, positive impact research can be picked up by the private sector more efficiently).
- Demanding that scientists come up with promises of positive impacts for their research compels them to make “integrity choices”. I am not saying that scientists lie, not even in the University of East Anglia, but the whole spirit of having to fill in blanks in a funding proposal encourages objectives creativity and scientific seduction.
- Focus on anachronistic impact assumptions disrupts the scientific process of discovery (eg, these results may be interesting, but I promised the EU that my research would produce these results, so I need to deliver on them and forego other potentially interesting avenues). Many great scientific discoveries are accidental, and our research resource management demands are taking away the ability to make ‘positive’ mistakes or chase butterflies.
- Impact demands mean that negative results often get discarded. There is a lot of research funding to show, for example, that global warming is occurring and that it is man-made. Results that do not confirm these funding requests are not considered valuable or publishable. How many false positives have been discarded?
- Tying research to the bottom line is a corporate endeavour. Scientists and academics are being corralled into a funding-based discovery process, forcing them to become venture scientists (aka, Craig Venter). Should academics be running SMEs or should they be generating research that others in the private sector can pick up and develop into innovations?
- When FP projects fail, often from my ex-post evaluation observations, I see cases where proposals had promised too much and failed to deliver, causing consortia to disintegrate. An FP consortium formed around a goal of proving X, Y or Z, that then cannot deliver what it promised, will face difficulties, even though L, M or N might be equally important, though unrealised.
- There is no real definition of positive impacts that evaluators can use as guidance. Solving world hunger? Stopping global warming? Curing major diseases with emerging technologies? Or rather more realistic approaches: improving learning and education opportunities? Building networks for potential cross-fertilisation, exchange and discovery? What impact is good enough, and how much is it worth? This is not objective, … not scientific.
As if seven reasons why we should change this approach in FP8 were not enough, my main reason to abandon impact-stressed research funding is that it leads to a loss of trust in science. The problem with green biotech and GMOs is not all of the scare stories and clever communications tricks that Greenpeace can conjure up, but rather the broken promises scientists had made. In the early 1990s, we accepted the risks of the unknown in biotech research, attracted to potential benefits like Golden Rice that would nourish the poor in Bangladesh, plants that could be grown in the desert and the reduction of the use of pesticides. These benefits did not immediately manifest – instead, we found ourselves with Roundup Ready corn and a feeling of betrayal. We believe what scientists promise us, but don’t understand that they are merely cataloguing potentials and that the market will innovate according to the best opportunities. Trust is lost and the ability for further funding in research encounters obstacles (GMO research in Europe suffered severe setbacks in the late 1990s). Other emerging sciences should have learnt from this trust crisis. Nanotech has provided great potential opportunities. Again, we are making scientists promise us great futures for nano research: easy delivery of cancer drugs, cures for major diseases, scratch-proof coatings … in the end, we get socks that make our feet stink less (and uncertainties they didn’t warn us about).
Stressing impact is a method some in the early Science and Society movement considered as important for restoring value and public trust in research after the risk crises of the 1990s. We should be counter-intuitive: talk less and do more. The best reactions people have to research advances are when they see the benefits (not when they are told about things they then expect but never get). So my first FP8 wish: find a way to get researchers to deliver more, but promise less – relegate impact to an add-on for any proposal.
It would be more cost-effective to give researchers money, let them keep their integrity, and see what they can produce in the long-term.David Zaruk