Do Data-Driven Meetings Work?

The use of data-driven meetings was popularized in the mid-1990s by the New York City police department, which dubbed them “CompStat” meetings.  These meetings were seen to contribute to a significant drop in crime and was eagerly replicated by other cities (Citi-Stat) and a number of states (State-Stat).  Harvard professor Bob Behn studied this phenomenon and wrote a book about “Performance-Stat” as a leadership style and way of thinking and behaving, not just an administrative process innovation.

At the federal level, data-driven meetings became more prevalent at the agency level beginning in 2009, according to a 2011 report by Harry Hatry and Elizabeth Davies of the Urban Institute.  Their use by federal agencies was reinforced by requirements in the GPRA Modernization Act of 2010, which the Government Accountability Office assessed in a 2015 report. Both the urban Institute and GAO reported positive effects and identified some best practices.

The spread of management innovations, like data-driven reviews, is oftentimes problematic in different policy, geographic, and cultural settings. Sometimes an innovation transfers well, other times, not so well.  But the spread of data-driven meetings quickly found an international following.

International Approaches. The United Kingdom created its own version of data-driven meetings in 2001, which were orchestrated by a newly-created organization reporting directly to Prime Minister Tony Blair. The Prime Minister’s Delivery Unit initially focused on improving citizen-facing services in four major policy areas – education, health, transportation, and crime.  Its first head, Michael Barber, subsequently wrote a book in 2008 about how the unit was set up and operated. His advice was similar to Dr. Behn’s –  using structured, data-informed processes are important, but the key to success is for the political leader to adopting the processes as the basis for a results-oriented, problem-solving leadership style.

The Delivery Unit concept spread rapidly in South America, beginning about five years ago.  A recent study by the Inter-American Development Bank assessed its use in 14 South American governments and concluded that, “under certain preconditions,” its use improved management and attained “results that have a direct impact on citizens.” For example, Chile and Peru focused on complex goals at the outcome level, such as reducing childhood malnutrition and increasing student math test scores.  Other countries, such as Costa Rico and Guatemala focused on delivering services such as the supply of medicines in hospitals or paving roads.

The study’s authors, Mariano Lafuente and Sebastián González, emphasize that the ultimate aim of delivery units “is to ensure that promises are kept.”

But achieving success wasn’t without institutional or political challenges. The authors’ survey of different delivery units across South America identified a series of challenges faced in implementing this approach.

Lessons Learned

Interestingly, the lessons learned in a review of the successes and challenges of the use of delivery units in South American governments have a familiar ring to them, since they parallel many of the lessons offered by Dr. Behn’s research on the state and local Performance-Stat approach:

• The chief executive likes the idea of a delivery unit but doesn’t devote the time to follow-up meetings nor delegates to someone who has the authority to deliver results.Top leaders must demonstrate ownership, note the authors: “If the highest political authority does not take ownership . . .by devoting time in quarterly or at least biannual follow up meetings . .. the model will simply not work.”
• The delivery unit is used by the chief executive for firefighting and not to achieve longer-term goals, and priorities are ill-defined or constantly changing.
• The delivery unit is introduced midway through a term of office. Typically, at that point in a term of office, central government positions are filled by other stakeholders and decisionmaking processes are already in place and difficult to change.
• Too much emphasis on collecting data to monitor gaps in performance, and not enough emphasis on learning and problem-solving.Creating scorecards to monitor progress and meet milestones doesn’t necessarily lead to achieving results. The goal should be to identify strategies for improvement and have mechanisms that allow rapid redeployment for resources to pursue the revised strategies.
• The head of the delivery unit lacks the necessary political profile (e.g., too academic and/or does not have influence or access to cabinet ministers).Or, the delivery unit team does not have the technical skills necessary to add value to the efforts being made by agencies.
• Other institutional actors feel threatened by a perceived intrusion into their roles and responsibilities and withhold their support.
• The line agencies do not see the delivery unit as a value-adding partner.It is critical to the success of delivery units to “be seen by them as a partner rather than a rival.” If not, the line agencies won’t share information with the delivery unit and will shut it out when trying to develop joint solutions to identified challenges. This includes working with budget teams to ensure resources available to tackle the challenges.
• Public reporting of results is always rosy, and credibility is lost in the eyes of the public. Political actors will begin to ignore the delivery unit as producing propaganda and the veracity of the data will be questioned.

Conclusion.  The authors conclude that the use of data-driven meetings using the delivery unit approach “are destined to become part of a wider ecosystem of public management innovations” that are used by central government agencies to improve decisionmaking, strengthen coordination between agencies and levels of government, and leverage better performance.  They believe this is the case not only because of the widespread adaptation of this approach by so many different kinds of governments across the globe, but also because digital transformations in government are allowing data to be easily collected, shared, and analyzed.

Graphic Credit: Courtesy of Sira Anamwong via FreeDigitalPhotos.net