Is It Possible to Have Value-Free Evidence?

In recent years, we have begun to see a number of scholars pointing towards a move away from more positivist understandings of science towards one, which more concertedly takes account of values in the scientific process (McMullin, 1982, Head, 2008). However, since the early 1990’s the rise of evidence-based policy has served as an important shift both within policy fields, but more generally in the role of science in society more generally.

The rise in the acceptance of values within the scientific process can be managed and integrated into an understanding of evidence-based policy. However, it does provide a key challenge, namely how policymakers integrate their values and ideologies, which inevitably privilege some research to the detriment of others. We can also see this challenge within the scientific process and indeed whether working towards value-free, objective research which in turn informs evidence, is at all possible, and more pointedly whether this challenge undermines the logic inherent within evidence-based policy. This is a question of not only which evidence but the objective and values within the production and development of an evidence base.  

Indeed, within (Cairney and Oliver, 2017) we can see that an interpretation of the role of evidence-based policy, must be integrated within the context of a more conventional understanding of the policymaking process, and acknowledge the political nature of policy, and in turn, the inherent value-driven approach which has come to dominate politics and policymaking more specifically. That politics differs significantly from the use of evidence-based interventions seen in medicine, should serve to contextualize the importance of values further, and highlight the importance of an understanding of values both with the production of evidence and indeed within its operationalization (Cairney and Oliver, 2017). This further highlights the problem of working towards a truly Value-free evidence-based policy, namely that “One may desire a political system based on value judgments and evidence, but should recognize and address the trade-offs between these aims, and that the production of evidence is also an inherently value-driven process.” This constitutes a core argument of this paper, namely that whilst one would expect to see truly objective evidence inform the policymaking process, the inherent disorder within the policymaking process makes such an intention much more fraught. This occurs when the logic and process-driven scientific approach, meet the disorderly policymaking process wherein trade-offs and values serve to shape and adapt evidence in the pursuit of particular aims. Through this, we can begin to see the challenge faced by evidence-based policy.

Trade-offs and compromises typically form a necessary lynchpin of the policymaking process. However, the challenge for the scientific community and researchers more generally when they seek to impact upon the policymaking process, is “that actors are influential when they ‘frame’ their evidence in simple, manipulative and/or emotional terms to generate policymaker attention” (Cairney and Oliver, 2017). This constitutes a key challenge to scientists, namely whether they should compromise upon the value of “scientific uncertainty” or indeed scientific skepticism, to instead focus on the crafting of more simplistic and absolutist claims to produce a more compelling rationale for policymakers to act, and whether this moves from evidence for intervention, towards evidence against intervention. This serves as a key intersection between the values within the scientific process, and the nature of the policymaking process. Cairney and Oliver (2017) also place the challenge of evidence-based policy with the context of the compromises and objectives of policy more generally. That evidence-based policy does not exist in a vacuum is important to contextualize the ways in which it can be used and interpreted. Whilst, for example, interest groups frequently play a role in the production of policy and can be used to build legitimacy effectively, or encourage buy-in from various stakeholders, the application of evidence can oftentimes be absolutist in terms of achieving particular objectives. This can instead focus on emphasizing the need for ‘fidelity’ and disempower local policy actors to applying the correct ‘dosage’ of an intervention to achieve set objectives (Cairney and Oliver, 2017). This can also be seen when competing bodies of evidence intersect, and value-based priorities are required to ensure that a socially optimum outcome is chosen and delivered upon. However, this can lead to a situation wherein a body of evidence is actively worked against in pursuit of differing objectives.

Indeed given the dominance and popularity of evidence-based policy, it can be seen that taking such a “rational-technical view of policymaking” (Pearce, Wesselink and Colebatch, 2014) assumes that a unified, agreed-upon body of evidence can in fact exist and that if policymakers simply paid more attention to implementing the evidence through policy, that better outcomes are a given. This logically flows back to the discussion on whether a) such a body of evidence can, in fact, exist, b) that such a body is based upon a consensus and c) that the evidence was produced through a value-neutral method and will be implemented as such. Whilst some interventions can be developed in such a way; it would be remiss to assume that all three criteria above can and often are satisfied to the point wherein it is simply political machinations, which stall the implementation of policy that is based on such an evidence body in the first place. Of particular interest is in whether it is possible to achieve a value-neutral evidence base at all, due to the centrality of scientific uncertainty in the scientific process. This also requires interpreting the policy process as an inherently logical and outcome-oriented approach, however as a number of scholars point to, this is often not the case, due in part to issues such as bounded rationality (Cairney and Oliver, 2017).

We can also see the political nature of policymaking affecting the use of evidence as outlined within (Head, 2008) wherein certain policy positions, become ‘data’ or ‘evidence-proof’ due to political commitments. This can lead to the use and application of evidence, which becomes politicized or delegitimized as “Some policy preferences allow only certain kinds of ‘evidence’ to be noticed and that “Critical commentary under these circumstances is unwelcome” (Head, 2008). There are many such examples, which point to headstrong policymakers becoming wedded to a particular policy, even though the best available evidence points to the contrary. An example we can think of to outline how this example can work in actuality is within energy policymaking, wherein certain bodies of evidence (for example the negative impact of carbon emissions on the environment or air quality for example) can be dismissed or the significance of this evidence be downplayed as other political priorities can take precedence. This can be due to a number of reasons such as assuaging voter bases in areas where there is a reliance on employment from environmentally damaging sectors (such as coal extraction or oil exploration). In this example, it is the privileging of one body of evidence (economic employment modeling for example) to the detriment of environmental evidence. That one body of evidence is used to counter the arguments used in another contradictory body of evidence involves a clear value judgment and the privileging of one body over another to achieve some particular objective. This can lead to such a policy intervention being considered “evidence-based” as it relied on evidence from one field and disregarded evidence from another. Politics and the policymaking process oftentimes require these value-based compromise positions, however, when it comes to particular bodies of research, there can oftentimes be a silo impact, wherein such compromise positions are rarely understood as delivering the most social optimum outcome given the particular challenges identified within the research. This oftentimes leads to lacking the insight to provide a value-judgment, given all variables. That these value judgments are used in tandem with evidence, indeed lends itself to an understanding of an evidence base. However, there are serious concerns, echoed by a number of scholars on how this evidence is developed and deployed in the policymaking process and causes the challenge of whether evidence bases are simply being used to legitimate particular policy choices.