 |
| See www.impactevaluation2011.org |
Mind the Gap: From Evidence to Policy Impact.
I liked the idea of dedicating 3 days, dozens of presentations and hundreds of blog posts to that little ‘leap of faith’ which characterises so many theories of change about what research can do for development.
The problem that we are faced with is that the normative idea about how policy should be made – based on objective evidence – is seldom the reality that we are faced with - i.e. policy through political expediency. Political expediency is understood to be a range of contextual influences on the decision-making process. When described this way, there is something inevitable about it.
Current thinking is that this expediency can be addressed through mediation of research knowledge. This has given rise to the research mediation sector- institutions and individuals within institutions who seek to frame research in a way that it is accessible and relevant to people working in key policy spheres.
What this reveals is a kind of contradiction at the heart of the development knowledge sector. While we call for evidence-based policy making, there is also increasing investment in the complex process that shapes decision making. A way through this may have been revealed through a closer look at what research mediation actually entails.
A couple years ago, IDS held a series of ‘influencing seminars’ which revealed how different disciplinary communities nuanced their approaches to policy influence depending on how they understood change happened. None of them declared disdain for value of quality evidence. Instead they all expressed differing views of what constitutes ‘quality’ evidence and how to gain traction with those who might need it.
What emerged was a framework of four different ways of building an effective relationship between research and quality policy making.
The first is about generating as many policy options as possible. This emphasises the use of repositories to allow users to sift through the options for themselves.
The second is evidence-based and prioritises the familiar idea that the quality of the research evidence is what will best inform the quality of the decision. Systematic reviews are seen as crucial in the research mediation process here.
Third is the value-led idea of policy-making. There are many examples of this leading to bad science, but it is by far the most common type of public policy making. Networks and epistemic communities are critical to the mediation process in this case.
Finally we have the relational model of influence, which maintains that no amount of research will influence a policymaker if there is not a relationship which reflects equity and a balance of power -where a researcher or a mediator are themselves subject to some influence.
Clearly though, none of these frames are mutually exclusive. Perhaps the point is that we can support the complex reality of policy influence which draws on these without losing sight of the where we ultimately need to get to. In fact using a little political expediency ourselves can go a long way to cross what is too often seen as a small gap.
Nick, thanks for highlighting/reminding us of the 'influencing' discussions we had at IDS a couple of years ago. It might be worth revisiting some of these ideas in light of recent events and developments (e.g. the power of crowd-sourcing and social media)?
ReplyDeleteI thought I might cheekily make the most of your reflections on 3ie's Mind the Gap conference to point to a blog post I wrote for the conference around the idea of "evaluation literacy" amongst those working to develop and implement policies and practices, entitled "Evaluation Literacy and reflections on the “Mind the Gap” conference" - http://www.impactevaluation2011.org/forum/?p=800
Impact evaluations provide a rigorous approach to assessing (broadly speaking) what works and what doesn't when it comes to policy or programme interventions. In terms of political expediency, clearly there should be a strong incentive to apply 'what works' (see, for example, the case of child nutrition in Peru http://ids.ac.uk/news/will-peru-sustain-progress-on-child-malnutrition) - but there is still a long way to go when it comes to translating the 'what works' (or doesn't) as established by research into ‘what can/ will work’, as achievable politically..
Perhaps we should be looking at what impact evaluations can learn from policy influence approaches, and what can this latter learn from impact evaluations. A systematic review of policy influencing approaches, anyone?
Nick - I have a variant on the relational model which is that technical experts in a field may well, for the most part, base their thinking on an issue around the current state of the evidence - in part because they are familiar with it as it is part of their expertise and they need this to help devise their own research.
ReplyDeletePolicy/decision makers on the other hand don't necessarily have the time or the skills to weigh up the "scientific" evidence across the range of topics they need to cover.
While packaging options and summarizing evidence can help convey this evidence, I believe that policy makers find a more effective way of sifting through both evidence and political feasibility (which include both politics but also cultural norms ideological beliefs, personal ambitions, social standing etc.) which is to take decisions based on some kind of arbitration process by considering the views of a wide and diverse network of people to help make a decision - this network would include both technical experts, but also implementers and people who can bring in the political dimensions which hep the decision maker balance between theoretically technically sound approaches, the policy makers own vision (which is likely to be at least partly ideological) and what is likely to be able to be feasible politically.
I think it would be interesting to understand more about how that decision making process works and what are some of the ways to identify good practices or learn from it in order to make this process more effective (understanding that evidence from research is only one of several valid sources of input to decision making).