How to "Mind the Gap"

I’ll never forget an incident early in my career, working in the Texas legislature, when a pair of academics testified on their statistical analysis for improving the funding distribution for school bus grants to school districts across the state.  They used regression analyses to demonstrate a more efficient and fairer way to allocate the aid.  But the state legislator was enraged – he asked how he was supposed to explain “this statistical mumbo-jumbo” (actually, his language was more colorful) to parents in his school district when they complained about why their kids couldn’t ride a bus to school.

Background.  It’s not just an academic vs. policy maker divide.  In looking back, the financial information generated by the Chief Financial Officers Act of 1990 and the performance information generated by the Government Performance and Results Act of 1993 have been judged to have little impact on policy decisions, especially in Congress.  The lesson: having technical experts in agencies creating a supply of information is not enough; there needed to be a demand for it, as well.

Up until recently, academics and scientists also felt that creating a supply and improving the quality of studies and research would be enough for policy makers to pay more attention to their work.  They thought that their goal was to reduce uncertainty, so policy makers could make more informed decisions.

Bridging a Culture Gap.  A new pair of articles in a recent issue of Public Administration Review shows that reducing uncertainty isn’t enough -- any more than it has been for financial or performance information.

In the first article, researchers Paul Cairney, Kathryn Oliver, Adam Wellstead say: “Improving the supply of evidence helps reduce scientific and policy maker uncertainty.  However, policy makers also combine their beliefs with limited evidence to reduce ambiguity” when choosing a course of action to solve a problem.  The researchers, therefore, explored whether there are solutions to closing the “evidence-policy” gap and “maximize the uptake of scientific evidence within government.”

They argue that academics and scientists who want their research used in policy making need to also factor in the wider process of debate, coalition building, and persuasion.  After all, policy makers want “to reduce ambiguity by establishing a dominant way to ‘frame’ policy problems.”  That is, having a point of view, or an ideological solution, simplifies decision making for them.  As a result, the researchers say it is important for advocates of scientific evidence to understand this and engage in the process of persuasion, as well, and “combine evidence with simple stories to exploit the emotional and ideological biases of policy makers.”

There are differences between academic and political cultures – language and jargon, longer scientific timescales, low incentives to engage, differing perceptions of scientific knowledge.  Scientists like to constantly challenge evidence to ensure it is robust; policy makers feel a need to create policy certainty and “reconcile evidence with well-established beliefs,” note the researchers.

The researchers think that a potential solution to this culture gap is to use the successful experiences of health and environmental scientists, who have learned to package their evidence in ways that are easy to understand. Oftentimes, they “foster networks with policy makers and encourage academic-practitioner workshops; use intermediaries such as ‘knowledge brokers;’ recognize the importance of timing; and encourage policy maker skills or better government understanding of problems.”  The article’s authors found that scientific knowledge only helps to reduce uncertainty. It does not address other dimensions that policy makers consider, ‘such as the inability to completely separate empirical facts from human values and ways of thinking.”

Using the Tools of Political Policy Experts.  The researchers suggest that academics and scientists adopt the tools of political policy theorists, where “The focus of policy theory is on the links between evidence, persuasion, and framing (in the wider context of a tendency for certain beliefs to dominate discussion).”

They offer the following four strategies to increase demand for scientific evidence:

• Combine facts with emotional appeals.
• Tell simple stories that are easy to understand.
• Interpret new evidence through the lens of the preexisting beliefs of actions.
• Offer policy solutions that are feasible.

They say that scientists have a difficult choice to make:  maintain an image of academic objectivity, or engage in advocacy to increase their impact.  They say: “Without such advocacy, policy makers will rely on highly biased and persuasive stories from vested interests rather than struggle through inaccessible and equivocal scientific papers.”

In concluding their article, the authors recommend academics and scientists make efforts to:

• promote networks between communities of scientists and policy makers to allow both sides to understand how the other camp operates and thinks.
• reward academics for doing policy-relevant research, and engage them in upcoming policy agenda-setting.
• use generalist ‘knowledge brokers’ who know who to ask which question, and have a good relationship with both policy makers and academics.
• measure the practical value of academic research on policy and policy making.

But There Is More.  In the second article, Max Arinder, an experienced government executive formerly with the Mississippi Joint Legislative Committee on Performance Evaluation and Expenditure Review, says that getting policy makers to use scientific evidence when making decisions isn’t a one-sided responsibility – both sides need to be open.  He notes: “ . . .experience tells me that there also needs to be a culture shift in the policy-making institutions themselves. Yes, policy makers often balance evidence, emotions, and beliefs in making a decision.”  The question is how to create an optimum balance.

In his experience working with state legislators, “Constituent opinion is a strong voice and will certainly be a part of any legislative arena. But how well informed is that voice?” (this reminded me of my Texas Legislature experience).  He says that “Just as constituent opinion is important in policy making, [a sound technical staff] is also an important element in determining the policy culture of a legislature.”

He observes that this “ . . . may depend on the will of policy makers to take the first steps in developing a transparent, evidence-informed policy process that increases public knowledge. . . .”  And he offers an example:  in 2015, Mississippi legislative leaders created a system to review requests for new programs and funding using an “evidence screen.” He says this shows that: “. . . we can make research-sensitive strategies a more important part of our core value system in the selection and funding of policy initiatives and programs.”  Similarly, the Pew Charitable Trusts is sponsoring similar efforts to expand the use of empirical evidence in other state legislatures.

In conclusion, both articles show how all stakeholders need to engage to shape a policy environment where the demand for evidence becomes part of the policy making culture.  While decisions may still be based on other considerations, the supporting data will be available and transparent – at a minimum hopefully create a better-informed electorate.

Graphic Credit:  Courtesy of WillMcC via Wikimedia Commons